-- Logs begin at Wed 2022-08-24 09:13:30 UTC, end at Wed 2022-08-24 10:50:24 UTC. -- Aug 24 09:54:40.502111 dv-cinder-854270-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 24 09:54:40.513481 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 09:54:40 2022] *** Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: nodename: dv-cinder-854270-2-iscsi Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: machine: x86_64 Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: clock source: unix Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: pcre jit disabled Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: detected number of CPU cores: 4 Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: current working directory: / Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: detected binary path: /usr/bin/uwsgi-core Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: your processes number limit is 47680 Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: your memory page size is 4096 bytes Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: detected max file descriptor number: 2048 Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: lock engine: pthread robust mutexes Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: thunder lock: enabled Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 24 09:54:40.514108 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 09:54:40.552422 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: Python main interpreter initialized at 0x55d370ac69c0 Aug 24 09:54:40.552422 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: python threads support enabled Aug 24 09:54:40.552422 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: your server socket listen backlog is limited to 100 connections Aug 24 09:54:40.552422 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: your mercy for graceful operations on workers is 90 seconds Aug 24 09:54:40.552814 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 09:54:40.552814 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: *** Operational MODE: preforking *** Aug 24 09:54:40.553084 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: *** uWSGI is running in multiple interpreter mode *** Aug 24 09:54:40.553084 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: spawned uWSGI master process (pid: 105446) Aug 24 09:54:40.553273 dv-cinder-854270-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 24 09:54:40.553820 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: spawned uWSGI worker 1 (pid: 105447, cores: 1) Aug 24 09:54:40.554154 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: spawned uWSGI worker 2 (pid: 105448, cores: 1) Aug 24 09:54:40.560184 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 09:54:41.210009 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 24 09:54:41.219968 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 24 09:54:42.295697 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:54:42.295697 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: from cryptography.utils import int_from_bytes Aug 24 09:54:42.296314 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:54:42.296314 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: from cryptography.utils import int_from_bytes Aug 24 09:54:42.300159 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:54:42.300159 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: from cryptography.utils import int_from_bytes Aug 24 09:54:42.300593 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:54:42.300593 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: from cryptography.utils import int_from_bytes Aug 24 09:54:44.799413 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:54:44.799907 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:54:44.800200 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:54:44.800561 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:54:44.800813 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:54:44.801225 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.801546 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.801867 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.802186 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.802514 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.802820 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.802911 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:54:44.803264 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:54:44.803346 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.803432 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.803599 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:54:44.803748 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.803964 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:54:44.804084 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.804248 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:54:44.804344 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.804637 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.804731 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-854270-2-iscsi {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.804952 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.805041 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.805258 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.805343 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.805568 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.805658 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.805884 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.805976 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.806187 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.806283 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.806501 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.806600 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.806818 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.806908 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.807127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.807295 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.807740 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.807740 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.807980 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.808123 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-854270-2-iscsi {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.808234 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.808343 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.808458 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.808641 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.808825 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.808950 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.809094 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.809230 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.809329 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.809534 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.809622 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.809856 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.809977 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.810162 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.810265 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.810465 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.810664 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.810862 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-854270-2-iscsi {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.810955 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.811174 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.811314 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.811483 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.811569 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.811794 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.811885 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.812211 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.812309 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.812521 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.812615 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.812826 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.812918 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.813133 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.813248 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.813452 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.813546 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.813757 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.813855 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.814081 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.814174 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-854270-2-iscsi {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.814399 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.814491 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.814717 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.814811 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.815034 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.815126 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.815335 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.815478 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.815640 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.815788 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.815943 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.816120 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.816285 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.816449 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.816586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.816818 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.816917 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.817128 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.817221 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.817433 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.817579 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.817764 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.817851 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.818083 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.818169 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.818402 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.818500 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.818651 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.818744 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.818900 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.818994 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.819240 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.819338 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.819558 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.819656 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.819862 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.819955 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.820187 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.820274 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.820437 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.820521 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.820743 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.820837 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.821066 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.821163 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.821366 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.821449 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.821687 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.821773 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.821971 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.822063 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.822220 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.822313 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.822540 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.822633 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.822903 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.822989 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.823214 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.823342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.823574 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.823672 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.823880 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.823964 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.86 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.824251 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.824338 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.86 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.824558 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.824731 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.824906 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.824991 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.825216 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.825311 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.825528 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.825614 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.825868 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.825953 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.826185 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.826269 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.826502 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.826631 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.826793 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.827122 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.827213 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.827445 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.827629 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.86 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.827809 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.827901 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.86 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.828161 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.828251 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.828413 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.828582 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.828712 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.828902 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.829015 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.829212 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.829331 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.829530 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.829634 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.829850 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.829946 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.830158 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.830244 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.830466 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.830552 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.830771 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.830862 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.831083 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.831174 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.831384 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.831470 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.831687 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.831771 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.831937 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.832021 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.832792 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.832884 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.833124 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.833218 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.833428 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.833513 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.833729 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.833813 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.834046 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.834138 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.834363 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.834447 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.834667 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.834753 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.834969 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.835065 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.835273 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.835356 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.835573 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.835658 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.835912 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.836010 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.836247 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.836333 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.836551 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.836636 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.836859 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.836964 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.837165 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.837259 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.837467 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.837556 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.837784 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.837878 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.838086 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.838171 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.838389 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.838474 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.838693 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.838785 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.838993 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.839077 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.839295 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.839386 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.839610 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.839702 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.839913 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.839997 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.840244 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.840368 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.840539 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.840639 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.840842 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.840928 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.841143 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.841235 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.841441 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.841526 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.841740 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.841832 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.842040 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.842126 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.842334 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.842419 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.842637 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.842721 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.842943 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.843034 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.843248 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.843340 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.843549 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.843633 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.843848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.843931 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.844176 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.844265 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.844480 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.844565 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.844782 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.844878 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.845085 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.845178 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.845387 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.845478 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.845687 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.845785 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.845984 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.846070 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.846299 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.846383 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.846616 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.846712 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.846923 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.847015 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.847227 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.847312 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.847532 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.847624 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.847874 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:54:44.848848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.849072 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.849178 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.849388 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.849511 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.849708 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.849793 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.850037 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.850124 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.850355 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.850449 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.850669 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.850754 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.850985 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.851070 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.851293 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.851391 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.851604 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.851776 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.851930 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.852015 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.852267 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.852361 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.852572 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.852657 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.852881 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.852973 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.853198 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.853291 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.853516 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.853641 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.853845 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.853943 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.854163 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.854253 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.854474 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.854566 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.854784 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.854917 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.855092 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.855178 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.855404 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.855499 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.855735 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.855834 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.856082 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.856210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.856409 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.856486 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.856722 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.856818 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.857033 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.857200 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.857377 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.857571 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.857716 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.857907 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.858029 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.858231 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.858349 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.858600 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.858697 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.858914 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.859041 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.859225 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.859310 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.859548 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.859642 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.859901 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.859986 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.860245 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.860343 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.860578 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.860672 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.860886 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.860972 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.861211 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.861296 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.861517 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.861616 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.861825 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.861939 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.862152 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.862241 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.862469 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.862580 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.862793 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.862900 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.863105 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.863280 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.863481 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.863659 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.863789 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.863983 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.864118 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.864319 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.864418 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.864633 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.864718 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.864944 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.865036 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.865251 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.865346 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.865558 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.865644 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.865865 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.865950 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.866176 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.866270 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.866483 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.866568 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.866785 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.866881 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.867108 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.867195 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.867412 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.867504 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.867715 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.867800 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.868027 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.868176 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.868359 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.868453 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.868669 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.868756 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.868981 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.869111 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.869286 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.869370 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.869600 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.869695 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.869908 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.869999 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.870214 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.870298 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.870542 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.870635 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.870856 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.870940 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.871156 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.871240 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.871478 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.871572 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.871808 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.871941 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.872140 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.872231 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.872446 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.872546 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.872754 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.872847 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.873056 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.873141 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.873359 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.873446 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.873659 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.873751 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.873968 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.874057 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.874271 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.874364 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.874572 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.874658 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.874873 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.874958 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.875175 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.875263 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.875476 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.875563 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.875796 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.875889 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.876127 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.876230 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.876431 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.876526 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.876735 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.876828 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.877036 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.877121 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.877339 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.877425 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.877648 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.877746 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.877959 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.878051 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.878270 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.878358 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.878576 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.878670 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.878880 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.878966 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.879210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.879294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.879515 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.879613 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.879819 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.879904 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.880169 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.880259 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.880480 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.880568 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.880786 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.880871 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.881091 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.881175 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.881395 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.881487 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.881710 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.881795 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.882020 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.882106 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.882342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.882429 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.882651 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.882742 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.882969 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.883054 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.883273 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.883359 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.883589 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.883673 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.883904 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.883990 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.884237 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.884325 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.884549 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.884633 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.884859 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.884962 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.885170 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.885262 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.885484 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.885574 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.885793 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.885885 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.886101 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.886184 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.886410 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.886502 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.886722 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.886815 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.887042 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.887128 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.887350 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.887442 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.887668 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.887762 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.887978 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.888134 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.888325 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.888506 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.888637 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.888884 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.888972 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.889259 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.889345 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.889571 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.889671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.889898 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.889983 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.890231 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.890323 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.890567 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.890667 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.890880 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.891014 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.891201 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.891406 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.891533 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.891749 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.891836 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.892139 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.892228 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.892458 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.892554 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.892776 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.892868 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.893085 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.893194 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.893397 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.893583 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.893744 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.893897 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.894053 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.894218 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.894357 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.894569 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.894693 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.895423 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.895532 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.895817 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.895903 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.896193 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.896292 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.896505 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.896590 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.896816 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.896901 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.897123 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.897214 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.897440 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.897531 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.897777 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.897870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.898105 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.898189 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.898429 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.898522 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.898751 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.898847 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.899058 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.899143 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.899366 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.899457 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.899670 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.899764 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.899972 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.900076 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.900301 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.900385 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.900619 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.900718 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.900926 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.901018 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.901228 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.901311 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.901532 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.901623 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.901838 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.901922 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.902147 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.902231 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.902455 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.902548 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.902763 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.902888 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.903086 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.903224 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.903387 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.903491 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.903683 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.903770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.903989 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.904136 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.904323 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.904411 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.904632 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.904719 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.904940 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.905035 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.905244 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.905329 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.905552 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.905636 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.905859 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.906021 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.906165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.906250 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.906465 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.906558 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.906771 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.906880 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.907097 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.907212 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.907397 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.907646 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.907732 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.907977 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.908098 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.908316 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.908411 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.908633 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.908734 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.908941 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.909026 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.909252 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.909338 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.909563 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.909662 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.909871 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.909956 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.910181 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.910274 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.911074 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.911195 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.911380 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.911472 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.911682 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.911768 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.911985 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.912104 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.912314 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.912400 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.912615 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.912700 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.912919 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.913006 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.913220 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.913480 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.913575 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.913813 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.913898 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.914112 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.914205 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.914414 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.914513 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.914714 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.914800 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.915021 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.915108 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.915320 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.915405 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.915626 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.915711 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.915927 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.916013 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.916271 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.916368 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.916578 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.916663 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.916878 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.916972 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.917181 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.917265 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.917481 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.917566 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.917822 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.917916 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.918126 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.918218 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.918464 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.918930 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.919014 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.919101 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.919101 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.919255 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.919394 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.919480 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.919694 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.919781 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.919997 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.920101 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.920330 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.920415 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.920632 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.920721 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.920932 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.921017 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.921234 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.921318 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.921538 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.921632 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.921837 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.921923 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.922140 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.922224 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.922446 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.922532 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.922749 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.922835 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.923053 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.923145 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.923351 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.923436 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.923657 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.923744 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.923956 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.924100 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.924313 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.924447 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.924626 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.924712 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.924949 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.925047 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.925252 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.925338 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.925561 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.925690 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.925892 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.925988 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.926230 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.926316 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.926540 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.926625 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.926862 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.926957 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.927191 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.927277 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.927492 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.927578 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.927816 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.927902 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.928148 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.928239 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.928467 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.928553 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.928777 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.928872 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.929102 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.929189 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.929429 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.929514 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.929737 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.929823 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.930041 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.930128 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.930367 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.930530 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.930684 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.930770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.931016 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.931108 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.931320 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.931413 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.931629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.931714 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.931944 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.932030 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.932274 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.932371 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.932580 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.932664 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.932883 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.932975 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.933183 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.933280 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.933485 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.933569 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.933783 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.933867 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.934100 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.934264 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.934391 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.934579 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.934689 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.934905 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.935000 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.935217 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.935303 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.935524 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.935610 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.935830 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.935924 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.936169 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.936260 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.936469 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.936565 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.936771 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.936863 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.937067 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.937199 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.937393 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.937479 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.937702 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.937786 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.938001 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.938085 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.938317 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.938404 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.938619 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.938706 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.938927 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.939011 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.939229 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.939313 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.939539 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.939677 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.939842 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.939938 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.940163 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.940250 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.940463 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.940557 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.940765 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.940850 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.941067 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.941165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.941367 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.941452 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.941916 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.942027 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.942117 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.942394 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.942481 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.942782 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.942869 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.943106 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.943192 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.943430 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.943516 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.943753 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.943838 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.944086 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.944310 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.944410 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.944626 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.944754 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.944942 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.945038 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.945260 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.945353 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.945571 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.945657 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.945884 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.945977 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.946191 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.946277 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.946495 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.946578 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.946800 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.946885 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.947109 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.947195 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.947413 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.947500 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.947714 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.947799 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.948018 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.948182 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.948363 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.948450 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.948684 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.948769 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.948992 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.949077 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.949298 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.949390 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.949610 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.949757 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.949908 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.949995 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.950214 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.950300 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.950515 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.950613 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.950898 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.950982 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.951210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.951295 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.951518 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.951613 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.951817 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.951902 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.952150 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.952240 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.952460 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.952570 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.952768 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.952854 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.953078 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.953163 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.953379 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.953465 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.953681 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.953796 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.953991 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.954106 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.954300 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.954412 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.954607 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.954718 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.954932 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.955020 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.955245 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.955338 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.955551 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.955635 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.955861 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.955993 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.956189 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.956279 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.956501 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.956586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.956798 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.956892 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.957117 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.957753 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.957975 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.958088 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.958273 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.958373 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.958656 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.958741 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.958994 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.959080 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.959296 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.959398 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.959639 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.959724 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.959954 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.960071 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.960322 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.960411 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.960692 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.960778 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.961014 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.961099 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.961334 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.961419 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.961668 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.961754 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.961989 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.962075 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.962298 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.962385 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.962647 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.962753 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.962966 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.963058 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.963299 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.963385 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.963612 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.963698 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.963940 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.964024 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.964292 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.964381 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.964636 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.964729 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.964983 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.965099 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.965311 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.965490 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.965722 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.965809 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.966045 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.966130 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.966354 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.966500 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.966676 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.966813 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.966998 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.967124 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.967302 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.967447 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.967605 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.967769 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.967911 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.968119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.968239 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.968426 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.968543 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.968738 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.968865 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.969048 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.969174 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.969378 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.969468 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.969703 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.969790 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.970013 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.970110 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.970327 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.970413 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.970631 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.970716 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.970950 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.971118 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.971252 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.971338 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.971653 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.971877 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.971970 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.972223 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.972310 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.972542 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.972642 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.972870 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.973747 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.974032 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.974182 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.974415 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.974517 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.974716 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.974798 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.975020 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.975106 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.975323 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.975409 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.975655 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.975741 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.975958 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.976078 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.976306 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.976394 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.976615 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.976703 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.976928 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.977013 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.977242 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.977341 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.977556 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.977641 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.977863 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.977950 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.978170 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.978268 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.978493 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.978578 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.978798 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.978884 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.979105 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.979198 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.979409 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.979493 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.979720 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.979806 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.980030 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.980191 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.980360 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.980447 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.980664 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.980750 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.980972 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.981059 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.981273 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.981358 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.981576 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.981673 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.981876 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.981962 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.982200 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.982293 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.982504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.982590 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.982824 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.982965 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.983130 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.983215 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.983432 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.983516 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.983737 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.983829 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.984038 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.984162 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.984401 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.984499 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.984706 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.984792 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.985032 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.985184 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.985341 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.985440 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.985650 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.985747 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.985956 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.986042 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.986268 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.986362 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.986571 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.986663 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.986879 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.986970 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.987185 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.987280 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.987494 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.987579 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.987798 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.987885 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.988125 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.988224 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.988430 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.988536 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.988768 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.988852 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.989072 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.989159 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.989400 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.989589 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.989710 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.989921 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.990016 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.990235 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.990322 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.990548 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.990633 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.990861 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.990956 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.991173 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.991266 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.991483 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.991576 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.991792 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.991892 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.992123 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.992212 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.992433 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.992527 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.992766 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.992853 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.993093 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.993178 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.993423 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.993510 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.993771 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.993858 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.994090 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.994176 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.994403 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.994497 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.994753 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.994845 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.995068 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.995153 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.995404 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.995497 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.995730 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.995822 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.996066 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.996274 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.996407 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.996730 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.996817 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.997059 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.997145 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.997394 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.997489 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.997707 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.997799 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.998036 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.998142 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.998353 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.998451 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.998664 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.998750 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.999036 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.999122 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.999347 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.999442 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.999660 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.999752 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:44.999978 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.000084 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.000325 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.000413 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.000652 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.000737 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.000975 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.001060 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.001287 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.001376 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.001599 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.001685 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.001911 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.001996 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.002223 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.002430 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.002577 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.002745 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.002892 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.003075 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.003207 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.003400 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.003486 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.003722 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.003807 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.004035 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.004691 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.004894 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.004983 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.005220 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.005307 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.005526 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.005620 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.005829 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.005921 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.006139 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.006233 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.006449 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.006540 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.006750 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.006841 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.007062 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.007155 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.007368 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.007453 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.007670 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.007756 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.007988 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.008105 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.008337 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.008422 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.008654 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.008866 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.008961 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.009216 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.009482 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.009607 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.009799 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.009885 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.010132 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.010218 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.010458 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.010543 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.010762 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.010854 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.011084 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.011244 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.011391 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.011484 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.011694 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.011780 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.012010 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.012115 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.012351 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.012509 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.012656 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.012761 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.012939 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.013035 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.013250 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.013336 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.013558 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.013653 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.013861 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.013954 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.014169 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.014255 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.014470 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.014568 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.014779 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.014877 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.015083 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.015315 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.015407 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.015665 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.015751 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.015969 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.016073 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.016301 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.016398 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.016604 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.016700 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.016903 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.016989 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.017204 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.017302 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.017512 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.017599 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.017816 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.017902 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.018124 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.018219 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.018471 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.018564 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.018735 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.018828 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.019036 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.019129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.019343 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.019429 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.019645 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.019751 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.020548 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.020638 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.020854 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.020946 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.021175 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.021303 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.021481 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.021579 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.021798 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.021882 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.022103 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.022186 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.022408 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.022494 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.022712 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.022805 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.023042 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.023133 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.023376 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.023468 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.023684 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.023918 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.024004 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.024307 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.024396 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.024630 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.024714 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.024934 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.025019 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.025288 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.025382 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.025593 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.025687 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.025929 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.026014 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.026235 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.026322 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.026544 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.026628 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.026941 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.027156 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.027242 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.027511 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.027598 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.027839 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.027992 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.028193 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.028338 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.028506 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.028689 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.028845 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.029014 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.029156 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.029333 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.029464 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.029646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.029786 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.030071 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.030158 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.030389 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.030525 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.030716 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.030928 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.031016 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.031268 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.031392 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.031595 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.031681 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.031932 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.032022 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.032276 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.032371 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.032590 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.032677 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.032896 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.032982 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.033203 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.033290 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.033507 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.033598 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.033815 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.033902 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.034122 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.034208 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.034430 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.034523 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.034741 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.034827 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.035055 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.035148 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.035384 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.035482 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.035792 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.035877 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.036122 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.036211 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.036438 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.036536 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.036751 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.036837 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.037057 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.037143 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.037366 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.037455 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.037677 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.037763 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.037984 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.038078 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.038294 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.038380 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.038601 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.038686 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.038924 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.039011 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.039243 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.039328 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.039565 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.039650 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.039883 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.039982 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.040210 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.040298 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.040515 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.040602 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.040817 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.040915 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.041123 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.041209 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.041434 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.041518 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.041740 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.041838 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.042041 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.042126 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.042342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.042427 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.042648 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.042764 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.042949 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.043078 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.043253 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.043481 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.043620 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.043806 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.043893 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.044141 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.044231 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.044446 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.044543 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.044744 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.044839 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.045043 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.045129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.045343 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.045438 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.045643 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.045737 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.045942 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.046027 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.046249 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.046335 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.046548 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.046633 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.046850 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.046944 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.047148 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.047235 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.047448 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.047540 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.047747 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.047832 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.048096 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.048186 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.048432 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.048518 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.048745 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.048842 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.049048 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.049133 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.049359 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.049444 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.049673 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.049768 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.049979 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.050065 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.050285 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.050377 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.050593 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.050687 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.050897 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.051002 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.051230 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.051316 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.051532 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.051625 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.051834 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.051920 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.052164 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.052252 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.052469 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.052557 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.052775 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.052861 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.053078 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.053163 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.053385 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.053471 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.053714 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.053806 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.054022 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.054108 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.054360 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.054446 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.054663 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.054755 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.054969 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.055054 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.055302 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.055395 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.055642 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.055729 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.055955 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.056076 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.056309 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.056393 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.056642 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.056735 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.056958 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.057052 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.057292 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.057377 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.057606 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.057692 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.057919 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.058053 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.058234 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.058318 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.058541 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.058625 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.058880 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.059010 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.059200 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.059370 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.059535 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.059685 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.059867 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.060062 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.060245 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.060379 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.060556 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.060781 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.060940 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.061120 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.061268 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.061476 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.061624 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.061805 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.061938 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.062114 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.062280 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.062440 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.062609 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.062759 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.062923 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.063082 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.063233 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.063398 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.063538 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.063709 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.063839 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105447) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:54:45.064058 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.064375 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.064687 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.064999 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.065311 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.065622 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.065938 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.066248 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.067071 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.067399 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.067719 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.068093 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.068399 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.068712 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.069041 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.069374 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.069723 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.070037 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.070379 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.070708 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.071026 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.071337 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.071644 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:54:45.071888 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:54:45.218562 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO dbcounter [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Registered counter for database nova_api Aug 24 09:54:45.227325 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO dbcounter [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Registered counter for database nova_api Aug 24 09:54:45.240942 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_db.sqlalchemy.engines [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105447) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:54:45.247554 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writer thread running {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:54:45.248146 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_db.sqlalchemy.engines [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105448) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:54:45.255415 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writer thread running {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:54:45.304348 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.context [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),62974e32-dd59-452e-afe9-bdb8e3ca6ae3(cell1) {{(pid=105447) load_cells /opt/stack/nova/nova/context.py:464}} Aug 24 09:54:45.308148 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_concurrency.lockutils [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105447) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 09:54:45.308627 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_concurrency.lockutils [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105447) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 09:54:45.310237 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_concurrency.lockutils [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105447) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 09:54:45.311018 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_concurrency.lockutils [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105447) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 09:54:45.311363 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.context [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),62974e32-dd59-452e-afe9-bdb8e3ca6ae3(cell1) {{(pid=105448) load_cells /opt/stack/nova/nova/context.py:464}} Aug 24 09:54:45.311472 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_concurrency.lockutils [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105447) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 09:54:45.313536 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_concurrency.lockutils [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105447) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 09:54:45.315474 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 09:54:45.315568 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO dbcounter [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Registered counter for database nova_cell0 Aug 24 09:54:45.316061 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 09:54:45.318304 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 09:54:45.319038 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 09:54:45.319487 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 09:54:45.321519 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 09:54:45.321633 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO dbcounter [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Registered counter for database nova_cell1 Aug 24 09:54:45.323506 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO dbcounter [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Registered counter for database nova_cell0 Aug 24 09:54:45.329319 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO dbcounter [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Registered counter for database nova_cell1 Aug 24 09:54:45.330869 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_db.sqlalchemy.engines [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105447) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:54:45.333799 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_db.sqlalchemy.engines [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105447) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:54:45.337333 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_db.sqlalchemy.engines [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105448) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:54:45.340943 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writer thread running {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:54:45.341319 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_db.sqlalchemy.engines [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105448) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:54:45.342479 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writer thread running {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:54:45.343294 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writer thread running {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:54:45.484759 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writer thread running {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:54:45.487232 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO dbcounter [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Registered counter for database nova_cell0 Aug 24 09:54:45.495175 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO dbcounter [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Registered counter for database nova_cell0 Aug 24 09:54:45.497689 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG oslo_db.sqlalchemy.engines [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105447) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:54:45.503743 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writer thread running {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:54:45.505577 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_db.sqlalchemy.engines [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105448) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:54:45.509846 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writer thread running {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:54:45.845534 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: WARNING keystonemiddleware.auth_token [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 24 09:54:45.853129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: WARNING keystonemiddleware.auth_token [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 24 09:54:45.858936 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: WARNING keystonemiddleware.auth_token [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 24 09:54:45.866628 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: WARNING keystonemiddleware.auth_token [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 24 09:54:46.194694 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: WARNING keystonemiddleware.auth_token [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 24 09:54:46.200854 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: WARNING keystonemiddleware.auth_token [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 24 09:54:46.206099 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d370ac69c0 pid: 105448 (default app) Aug 24 09:54:46.208211 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:54:46.209413 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] 10.222.0.86 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001766 Aug 24 09:54:46.209832 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 1/1] 10.222.0.86 () {50 vars in 788 bytes} [Wed Aug 24 09:54:46 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:54:46.222055 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: WARNING keystonemiddleware.auth_token [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 24 09:54:46.228386 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: WARNING keystonemiddleware.auth_token [None req-e65d62bb-0da0-49b5-90df-f6a17af79e99 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 24 09:54:46.233670 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d370ac69c0 pid: 105447 (default app) Aug 24 09:56:04.491929 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:04.492975 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001497 Aug 24 09:56:04.493537 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 2/2] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:04.505798 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writing DB stats nova_api:SELECT=3 {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:04.506917 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writing DB stats nova_cell0:SELECT=3 {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:04.507825 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writing DB stats nova_cell1:SELECT=3 {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:04.509306 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:05.384627 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-aca0a925-36f9-4d02-a76e-64900c389f31 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:05.417804 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-aca0a925-36f9-4d02-a76e-64900c389f31 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.917327 Aug 24 09:56:05.418706 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 1/3] 10.222.0.86 () {60 vars in 1292 bytes} [Wed Aug 24 09:56:04 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 919 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:07.853397 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:07.854641 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-9fd8719e-ccc6-47e0-ae33-502b13825af7 None None] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001759 Aug 24 09:56:07.855362 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 3/4] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:07 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:08.322883 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-176f6db3-358d-442e-980e-0cc657a5d716 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "ram": 256, "vcpus": 1, "rxtx_factor": 1.0, "name": "cirros256", "id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "disk": 1}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:08.346211 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:08.346211 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: warnings.warn(msg) Aug 24 09:56:08.378520 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-176f6db3-358d-442e-980e-0cc657a5d716 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.513191 Aug 24 09:56:08.379363 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 2/5] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:07 2022] POST /compute/v2.1/flavors => generated 407 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:56:08.394049 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_api:SELECT=3 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:08.395206 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_cell0:SELECT=3 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:08.396280 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_cell1:SELECT=3 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:08.397388 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:08.607069 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-fc0ecfb6-1565-4249-97ba-29127cdced4f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:08.611648 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:08.611648 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: warnings.warn(msg) Aug 24 09:56:08.671735 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-fc0ecfb6-1565-4249-97ba-29127cdced4f admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.283758 Aug 24 09:56:08.672634 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 4/6] 10.222.0.86 () {64 vars in 1336 bytes} [Wed Aug 24 09:56:08 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:11.084240 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-176f6db3-358d-442e-980e-0cc657a5d716 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:11.085266 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-176f6db3-358d-442e-980e-0cc657a5d716 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Aug 24 09:56:11.085947 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 3/7] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:11.775818 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-7de74299-0e8d-4d3f-8210-162e794a3b0e admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 5, "os-flavor-access:is_public": true, "vcpus": 1, "ram": 512, "name": "ds512M", "rxtx_factor": 1.0, "id": "d1", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:11.811880 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:11.811880 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: warnings.warn(msg) Aug 24 09:56:11.834242 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-7de74299-0e8d-4d3f-8210-162e794a3b0e admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.742016 Aug 24 09:56:11.836467 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 5/8] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:11 2022] POST /compute/v2.1/flavors => generated 404 bytes in 745 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:56:11.847641 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-6eb9fd75-eb72-49d7-b380-7523f39d9e27 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:11.850977 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:11.850977 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: warnings.warn(msg) Aug 24 09:56:11.887928 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-6eb9fd75-eb72-49d7-b380-7523f39d9e27 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.044471 Aug 24 09:56:11.895852 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 4/9] 10.222.0.86 () {64 vars in 1336 bytes} [Wed Aug 24 09:56:11 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:14.291264 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-7de74299-0e8d-4d3f-8210-162e794a3b0e admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:14.292432 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-7de74299-0e8d-4d3f-8210-162e794a3b0e admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001581 Aug 24 09:56:14.292928 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 6/10] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:14 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:14.698833 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-c5e71491-80bf-491f-a558-adbca2642ae1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "name": "ds1G", "id": "d2", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "disk": 10, "rxtx_factor": 1.0, "swap": 0, "os-flavor-access:is_public": true}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:14.718799 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-c5e71491-80bf-491f-a558-adbca2642ae1 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.419002 Aug 24 09:56:14.719791 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 5/11] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:14 2022] POST /compute/v2.1/flavors => generated 404 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:56:14.731823 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-3414f399-8116-4405-af32-683e67bcf491 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:14.755973 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-3414f399-8116-4405-af32-683e67bcf491 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027811 Aug 24 09:56:14.756588 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 7/12] 10.222.0.86 () {64 vars in 1336 bytes} [Wed Aug 24 09:56:14 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:17.118002 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-c5e71491-80bf-491f-a558-adbca2642ae1 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:17.119010 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-c5e71491-80bf-491f-a558-adbca2642ae1 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002074 Aug 24 09:56:17.119801 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 6/13] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:17 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:17.562584 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-1b6cc253-de86-4348-a8b2-8463430e30e0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 10, "id": "d3", "name": "ds2G", "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "ram": 2048}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:17.583557 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-1b6cc253-de86-4348-a8b2-8463430e30e0 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.457675 Aug 24 09:56:17.591282 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 8/14] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:17 2022] POST /compute/v2.1/flavors => generated 404 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:56:17.596309 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-79cd6ba2-6d97-4da3-bbee-b2616139a654 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:17.625980 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-79cd6ba2-6d97-4da3-bbee-b2616139a654 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028817 Aug 24 09:56:17.625980 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 7/15] 10.222.0.86 () {64 vars in 1336 bytes} [Wed Aug 24 09:56:17 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:20.215961 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-1b6cc253-de86-4348-a8b2-8463430e30e0 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:20.216913 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-1b6cc253-de86-4348-a8b2-8463430e30e0 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Aug 24 09:56:20.217317 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 9/16] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:20.662116 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-97f1d5aa-a54a-4bc1-9e9c-86bb57be5375 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds4G", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "id": "d4", "vcpus": 4, "rxtx_factor": 1.0, "ram": 4096, "os-flavor-access:is_public": true, "disk": 20}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:20.683889 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-97f1d5aa-a54a-4bc1-9e9c-86bb57be5375 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.460326 Aug 24 09:56:20.685091 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 8/17] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:20 2022] POST /compute/v2.1/flavors => generated 404 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 09:56:20.697584 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-9056e23b-9b08-4876-af61-e316b1b99bc7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:20.724090 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-9056e23b-9b08-4876-af61-e316b1b99bc7 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030622 Aug 24 09:56:20.725035 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 10/18] 10.222.0.86 () {64 vars in 1336 bytes} [Wed Aug 24 09:56:20 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:23.592883 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-97f1d5aa-a54a-4bc1-9e9c-86bb57be5375 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:23.593948 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-97f1d5aa-a54a-4bc1-9e9c-86bb57be5375 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001567 Aug 24 09:56:23.594908 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 9/19] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:23 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:24.034199 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-45d6b4c4-17e1-4639-ba9b-53ffd493c5d0 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:24.058867 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-45d6b4c4-17e1-4639-ba9b-53ffd493c5d0 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.457644 Aug 24 09:56:24.059434 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 11/20] 10.222.0.86 () {60 vars in 1292 bytes} [Wed Aug 24 09:56:23 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 09:56:26.541026 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-97f1d5aa-a54a-4bc1-9e9c-86bb57be5375 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:26.542386 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-97f1d5aa-a54a-4bc1-9e9c-86bb57be5375 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002047 Aug 24 09:56:26.542948 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 10/21] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:26 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:26.972134 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-d8ba9609-d9bc-410d-98cc-430d0f4db48e admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "id": "1", "os-flavor-access:is_public": true, "ram": 512, "disk": 1, "swap": 0, "vcpus": 1, "name": "m1.tiny", "rxtx_factor": 1.0}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:26.998148 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-d8ba9609-d9bc-410d-98cc-430d0f4db48e admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.446619 Aug 24 09:56:26.999286 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 12/22] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:26 2022] POST /compute/v2.1/flavors => generated 402 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:56:27.011935 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-ecfb450e-fdc8-4536-b744-b01b19e325c6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:27.036411 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-ecfb450e-fdc8-4536-b744-b01b19e325c6 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028296 Aug 24 09:56:27.036993 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 11/23] 10.222.0.86 () {64 vars in 1333 bytes} [Wed Aug 24 09:56:27 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 09:56:29.378113 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-d8ba9609-d9bc-410d-98cc-430d0f4db48e admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:29.379148 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-d8ba9609-d9bc-410d-98cc-430d0f4db48e admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 Aug 24 09:56:29.379549 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 13/24] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:29 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:29.756192 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-a05a5605-0588-4195-9e42-2e86aa85af1e admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "2", "disk": 20, "name": "m1.small", "os-flavor-access:is_public": true, "vcpus": 1, "ram": 2048, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:29.776992 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-a05a5605-0588-4195-9e42-2e86aa85af1e admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.391064 Aug 24 09:56:29.778268 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 12/25] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:29 2022] POST /compute/v2.1/flavors => generated 405 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:56:29.791791 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-137cafc5-c92e-488a-8608-e605b8b8d2b8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:29.818305 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-137cafc5-c92e-488a-8608-e605b8b8d2b8 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030403 Aug 24 09:56:29.818882 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 14/26] 10.222.0.86 () {64 vars in 1333 bytes} [Wed Aug 24 09:56:29 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:32.249012 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-a05a5605-0588-4195-9e42-2e86aa85af1e admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:32.250051 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-a05a5605-0588-4195-9e42-2e86aa85af1e admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001494 Aug 24 09:56:32.250494 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 13/27] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:32 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:32.747485 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-3bbfaef9-0eb7-4b34-8423-e71445673c97 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "3", "disk": 40, "name": "m1.medium", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "vcpus": 2, "ram": 4096, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:32.770188 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-3bbfaef9-0eb7-4b34-8423-e71445673c97 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.513198 Aug 24 09:56:32.770854 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 15/28] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:32 2022] POST /compute/v2.1/flavors => generated 406 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:56:32.784324 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-71217556-1615-49ec-a669-9ac502319249 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:32.810298 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-71217556-1615-49ec-a669-9ac502319249 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030848 Aug 24 09:56:32.811013 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 14/29] 10.222.0.86 () {64 vars in 1333 bytes} [Wed Aug 24 09:56:32 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:35.200739 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-3bbfaef9-0eb7-4b34-8423-e71445673c97 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:35.201946 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-3bbfaef9-0eb7-4b34-8423-e71445673c97 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001716 Aug 24 09:56:35.202454 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 16/30] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:35.646174 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-859f3aad-6cde-4e27-b6b0-43baf23dbd8e admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 8192, "vcpus": 4, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "m1.large", "id": "4", "rxtx_factor": 1.0, "disk": 80}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:35.651696 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:35.670062 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-859f3aad-6cde-4e27-b6b0-43baf23dbd8e admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.460707 Aug 24 09:56:35.670655 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 15/31] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:35 2022] POST /compute/v2.1/flavors => generated 405 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 09:56:35.683254 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-171a8d10-3ca7-407b-b94d-98669ec87832 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:35.707287 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-171a8d10-3ca7-407b-b94d-98669ec87832 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027565 Aug 24 09:56:35.707777 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 17/32] 10.222.0.86 () {64 vars in 1333 bytes} [Wed Aug 24 09:56:35 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:38.201259 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-859f3aad-6cde-4e27-b6b0-43baf23dbd8e admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:38.202876 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-859f3aad-6cde-4e27-b6b0-43baf23dbd8e admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002057 Aug 24 09:56:38.204094 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 16/33] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:38 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:38.661922 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-25bf98ef-e0e9-4afd-b11d-0e1822772828 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "ram": 16384, "disk": 160, "vcpus": 8, "rxtx_factor": 1.0, "id": "5", "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.xlarge", "os-flavor-access:is_public": true}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:38.668560 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:38.693508 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-25bf98ef-e0e9-4afd-b11d-0e1822772828 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.482623 Aug 24 09:56:38.694361 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 18/34] 10.222.0.86 () {64 vars in 1283 bytes} [Wed Aug 24 09:56:38 2022] POST /compute/v2.1/flavors => generated 408 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:56:38.708290 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-e46f6dd1-83f7-498c-a260-a8cdbd3c94b8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:56:38.738321 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-e46f6dd1-83f7-498c-a260-a8cdbd3c94b8 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035098 Aug 24 09:56:38.738898 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 17/35] 10.222.0.86 () {64 vars in 1333 bytes} [Wed Aug 24 09:56:38 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:56:56.639637 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-25bf98ef-e0e9-4afd-b11d-0e1822772828 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:56.641147 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-25bf98ef-e0e9-4afd-b11d-0e1822772828 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002033 Aug 24 09:56:56.641547 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 19/36] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:56:56 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:56.651919 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:57.045803 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-a6b9a205-0884-40ff-86c2-c31992697ea4 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:57.065068 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-a6b9a205-0884-40ff-86c2-c31992697ea4 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.417583 Aug 24 09:56:57.065593 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 18/37] 10.222.0.86 () {60 vars in 1292 bytes} [Wed Aug 24 09:56:56 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 09:56:58.114756 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-25bf98ef-e0e9-4afd-b11d-0e1822772828 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:58.115768 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-25bf98ef-e0e9-4afd-b11d-0e1822772828 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001430 Aug 24 09:56:58.116202 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 20/38] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 09:56:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:56:58.635948 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-9c67410d-da35-4c47-8041-39da22726dbd admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:58.641554 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.wsgi [None req-9c67410d-da35-4c47-8041-39da22726dbd admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-9c67410d-da35-4c47-8041-39da22726dbd admin admin] Request method failure captured: Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Accept: application/json Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Accept-Encoding: gzip, deflate Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Connection: keep-alive Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Content-Length: 0 Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Host: 10.222.0.86 Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Openstack-Api-Version: compute 2.90 Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Openstack-System-Scope: None Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: User-Agent: python-novaclient Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Auth-Token: gAAAAABjBfXqnGjS9CzFENbQqZaLtFLyPXVEB6dL4o99hXjMN4mzOgSj9PcSIf1LKw4gLFtAX2IDQjCXswBAP3hlf1HY4LN-5NqgYXHeEY_CHjkXFZrePm3W-z13i3cHpJ27P0C6ZCOeUv_Fva6cwd1DRwFCXrvoltbeLQr5srKfvlORRLNauDA Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Domain-Id: None Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Domain-Name: None Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Identity-Status: Confirmed Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Is-Admin-Project: True Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Openstack-Nova-Api-Version: 2.90 Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Domain-Id: default Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Domain-Name: Default Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Name: admin Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Role: reader,admin,member Aug 24 09:56:58.644823 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Roles: reader,admin,member Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f71a167399734714a60b777eea4e2ecc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Tenant: admin Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Tenant-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Tenant-Name: admin Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User: admin Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Domain-Id: default Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Domain-Name: Default Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Id: 9757335c28f74ea5a8fbdedc0ae7682a Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Name: admin Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: method: > Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: exception: Flavor 42 could not be found. Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: action_args: {'id': '42'} Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 09:56:58.646504 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:56:58.647728 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.649165 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-9c67410d-da35-4c47-8041-39da22726dbd admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105447) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 09:56:58.650483 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-9c67410d-da35-4c47-8041-39da22726dbd admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.392170 Aug 24 09:56:58.650483 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 19/39] 10.222.0.86 () {60 vars in 1190 bytes} [Wed Aug 24 09:56:58 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 393 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:58.655900 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-e9174ca9-9946-4671-91cc-7cd0d0ed8a35 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:58.679173 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-e9174ca9-9946-4671-91cc-7cd0d0ed8a35 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026727 Aug 24 09:56:58.679643 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 21/40] 10.222.0.86 () {60 vars in 1210 bytes} [Wed Aug 24 09:56:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 09:56:58.689943 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-068ba837-d0b6-4fad-aa00-b926a214b7fe admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:58.713378 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-068ba837-d0b6-4fad-aa00-b926a214b7fe admin admin] 10.222.0.86 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027584 Aug 24 09:56:58.713935 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 20/41] 10.222.0.86 () {60 vars in 1210 bytes} [Wed Aug 24 09:56:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 09:56:58.724240 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-4a1ecdda-33cf-4d5d-9aa2-f451aa6e9765 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:58.729915 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.wsgi [None req-4a1ecdda-33cf-4d5d-9aa2-f451aa6e9765 admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-4a1ecdda-33cf-4d5d-9aa2-f451aa6e9765 admin admin] Request method failure captured: Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Accept: application/json Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Accept-Encoding: gzip, deflate Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Connection: keep-alive Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Content-Length: 0 Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Host: 10.222.0.86 Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Openstack-Api-Version: compute 2.90 Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Openstack-System-Scope: None Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: User-Agent: python-novaclient Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Auth-Token: gAAAAABjBfXqnGjS9CzFENbQqZaLtFLyPXVEB6dL4o99hXjMN4mzOgSj9PcSIf1LKw4gLFtAX2IDQjCXswBAP3hlf1HY4LN-5NqgYXHeEY_CHjkXFZrePm3W-z13i3cHpJ27P0C6ZCOeUv_Fva6cwd1DRwFCXrvoltbeLQr5srKfvlORRLNauDA Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Domain-Id: None Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Domain-Name: None Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Identity-Status: Confirmed Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Is-Admin-Project: True Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Openstack-Nova-Api-Version: 2.90 Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Domain-Id: default Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Domain-Name: Default Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Name: admin Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Role: reader,admin,member Aug 24 09:56:58.733119 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Roles: reader,admin,member Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f71a167399734714a60b777eea4e2ecc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Tenant: admin Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Tenant-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Tenant-Name: admin Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User: admin Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Domain-Id: default Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Domain-Name: Default Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Id: 9757335c28f74ea5a8fbdedc0ae7682a Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Name: admin Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: method: > Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: exception: Flavor 42 could not be found. Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: action_args: {'id': '42'} Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 09:56:58.735129 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:56:58.736238 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:56:58.737870 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-4a1ecdda-33cf-4d5d-9aa2-f451aa6e9765 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105448) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 09:56:58.739127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-4a1ecdda-33cf-4d5d-9aa2-f451aa6e9765 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.015242 Aug 24 09:56:58.739127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 22/42] 10.222.0.86 () {60 vars in 1190 bytes} [Wed Aug 24 09:56:58 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:59.696493 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-068ba837-d0b6-4fad-aa00-b926a214b7fe admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:56:59.697509 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-068ba837-d0b6-4fad-aa00-b926a214b7fe admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Aug 24 09:56:59.697905 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 21/43] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 09:56:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:57:00.221795 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-c7327232-7136-483c-b919-ac5481898a47 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:00.227141 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.wsgi [None req-c7327232-7136-483c-b919-ac5481898a47 admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-c7327232-7136-483c-b919-ac5481898a47 admin admin] Request method failure captured: Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Accept: application/json Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Accept-Encoding: gzip, deflate Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Connection: keep-alive Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Content-Length: 0 Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Host: 10.222.0.86 Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Openstack-Api-Version: compute 2.90 Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Openstack-System-Scope: None Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: User-Agent: python-novaclient Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Auth-Token: gAAAAABjBfXrUz7r3k4v9eKBgyg-K0IWFar6nJQRQEb3mllLfroOJECOnj4HkHXT2KrA5HqYlHXcNrzp9tnUs3FF6d_Fr8e1QMYsEdB4ZfNelP8wluKjyV-EEtTVWisNbf7XkkyIJKYn20bmG_wMJIPM64Icb1MlH2vVbRssoCyHEHCr_ufZUNE Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Domain-Id: None Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Domain-Name: None Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Identity-Status: Confirmed Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Is-Admin-Project: True Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Openstack-Nova-Api-Version: 2.90 Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Domain-Id: default Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Domain-Name: Default Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Name: admin Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Role: reader,admin,member Aug 24 09:57:00.228671 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Roles: reader,admin,member Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f71a167399734714a60b777eea4e2ecc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Tenant: admin Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Tenant-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Tenant-Name: admin Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User: admin Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Domain-Id: default Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Domain-Name: Default Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Id: 9757335c28f74ea5a8fbdedc0ae7682a Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Name: admin Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: method: > Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: exception: Flavor 84 could not be found. Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: action_args: {'id': '84'} Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 09:57:00.230232 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:57:00.231210 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.232646 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-c7327232-7136-483c-b919-ac5481898a47 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105448) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 09:57:00.234475 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-c7327232-7136-483c-b919-ac5481898a47 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.394454 Aug 24 09:57:00.234475 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 23/44] 10.222.0.86 () {60 vars in 1190 bytes} [Wed Aug 24 09:56:59 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 395 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:00.240487 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-c13a7e18-8b4c-4c66-9ff0-92d6882ac26e admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:00.255809 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-c13a7e18-8b4c-4c66-9ff0-92d6882ac26e admin admin] 10.222.0.86 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019117 Aug 24 09:57:00.256411 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 22/45] 10.222.0.86 () {60 vars in 1210 bytes} [Wed Aug 24 09:57:00 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 09:57:00.266172 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-af75fd36-465a-4b3c-a982-bd9f1f2deddd admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:00.281067 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-af75fd36-465a-4b3c-a982-bd9f1f2deddd admin admin] 10.222.0.86 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018822 Aug 24 09:57:00.281591 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 24/46] 10.222.0.86 () {60 vars in 1210 bytes} [Wed Aug 24 09:57:00 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 09:57:00.290897 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-242f7217-86a1-4812-8102-3ce5538df05f admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:00.296848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.wsgi [None req-242f7217-86a1-4812-8102-3ce5538df05f admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-242f7217-86a1-4812-8102-3ce5538df05f admin admin] Request method failure captured: Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Accept: application/json Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Accept-Encoding: gzip, deflate Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Connection: keep-alive Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Content-Length: 0 Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Host: 10.222.0.86 Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Openstack-Api-Version: compute 2.90 Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Openstack-System-Scope: None Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: User-Agent: python-novaclient Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Auth-Token: gAAAAABjBfXrUz7r3k4v9eKBgyg-K0IWFar6nJQRQEb3mllLfroOJECOnj4HkHXT2KrA5HqYlHXcNrzp9tnUs3FF6d_Fr8e1QMYsEdB4ZfNelP8wluKjyV-EEtTVWisNbf7XkkyIJKYn20bmG_wMJIPM64Icb1MlH2vVbRssoCyHEHCr_ufZUNE Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Domain-Id: None Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Domain-Name: None Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Identity-Status: Confirmed Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Is-Admin-Project: True Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Openstack-Nova-Api-Version: 2.90 Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Domain-Id: default Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Domain-Name: Default Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Name: admin Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Role: reader,admin,member Aug 24 09:57:00.298342 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Roles: reader,admin,member Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f71a167399734714a60b777eea4e2ecc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Tenant: admin Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Tenant-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Tenant-Name: admin Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User: admin Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Domain-Id: default Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Domain-Name: Default Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Id: 9757335c28f74ea5a8fbdedc0ae7682a Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Name: admin Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: method: > Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: exception: Flavor 84 could not be found. Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: action_args: {'id': '84'} Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 09:57:00.299848 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:57:00.300821 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:00.302294 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-242f7217-86a1-4812-8102-3ce5538df05f admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105447) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 09:57:00.303580 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-242f7217-86a1-4812-8102-3ce5538df05f admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012383 Aug 24 09:57:00.303580 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 23/47] 10.222.0.86 () {60 vars in 1190 bytes} [Wed Aug 24 09:57:00 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Aug 24 09:57:01.259852 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-af75fd36-465a-4b3c-a982-bd9f1f2deddd admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:01.260894 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-af75fd36-465a-4b3c-a982-bd9f1f2deddd admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001498 Aug 24 09:57:01.261296 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 25/48] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 09:57:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:57:01.778533 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-26e3a1cd-e235-46a6-9bdc-19e5d0f6b7d5 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:01.784934 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.wsgi [None req-26e3a1cd-e235-46a6-9bdc-19e5d0f6b7d5 admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-26e3a1cd-e235-46a6-9bdc-19e5d0f6b7d5 admin admin] Request method failure captured: Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Accept: application/json Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Accept-Encoding: gzip, deflate Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Connection: keep-alive Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Content-Length: 0 Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Host: 10.222.0.86 Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Openstack-Api-Version: compute 2.90 Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Openstack-System-Scope: None Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: User-Agent: python-novaclient Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Auth-Token: gAAAAABjBfXt2urC6hNXaE0tcpX7CKm91T1ajRvpaZzelf__0ImFevVpI_DEosKc1WHY6ZGj3hShnFM-QhK_ICrmhBcx8AVlUqLcARIHj_m2nfX7U4iD6khM3twhz5A0kNGJMg_vMicoG4baVc1lMQXcpeXi4XjKk0J5LtrMRMmNOVlPtcfIIWo Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Domain-Id: None Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Domain-Name: None Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Identity-Status: Confirmed Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Is-Admin-Project: True Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Openstack-Nova-Api-Version: 2.90 Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Domain-Id: default Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Domain-Name: Default Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Project-Name: admin Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Role: reader,admin,member Aug 24 09:57:01.790090 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Roles: reader,admin,member Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f71a167399734714a60b777eea4e2ecc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Tenant: admin Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Tenant-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-Tenant-Name: admin Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User: admin Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Domain-Id: default Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Domain-Name: Default Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Id: 9757335c28f74ea5a8fbdedc0ae7682a Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: X-User-Name: admin Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: method: > Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: exception: Flavor 451 could not be found. Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: action_args: {'id': '451'} Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 09:57:01.791652 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:57:01.792586 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.793629 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-26e3a1cd-e235-46a6-9bdc-19e5d0f6b7d5 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105447) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 09:57:01.794733 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-26e3a1cd-e235-46a6-9bdc-19e5d0f6b7d5 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.394477 Aug 24 09:57:01.794733 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 24/49] 10.222.0.86 () {60 vars in 1193 bytes} [Wed Aug 24 09:57:01 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 396 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:01.801047 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-85fb07f1-106d-44bd-8853-93048d217736 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:01.816165 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-85fb07f1-106d-44bd-8853-93048d217736 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018603 Aug 24 09:57:01.816688 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 26/50] 10.222.0.86 () {60 vars in 1210 bytes} [Wed Aug 24 09:57:01 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 09:57:01.825890 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:01.841147 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018759 Aug 24 09:57:01.841671 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 25/51] 10.222.0.86 () {60 vars in 1210 bytes} [Wed Aug 24 09:57:01 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 09:57:01.850864 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-d57da1f1-865e-484c-830a-32adc8d42cba admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:01.856909 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.wsgi [None req-d57da1f1-865e-484c-830a-32adc8d42cba admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-d57da1f1-865e-484c-830a-32adc8d42cba admin admin] Request method failure captured: Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Accept: application/json Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Accept-Encoding: gzip, deflate Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Connection: keep-alive Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Content-Length: 0 Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Host: 10.222.0.86 Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Openstack-Api-Version: compute 2.90 Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Openstack-System-Scope: None Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: User-Agent: python-novaclient Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Auth-Token: gAAAAABjBfXt2urC6hNXaE0tcpX7CKm91T1ajRvpaZzelf__0ImFevVpI_DEosKc1WHY6ZGj3hShnFM-QhK_ICrmhBcx8AVlUqLcARIHj_m2nfX7U4iD6khM3twhz5A0kNGJMg_vMicoG4baVc1lMQXcpeXi4XjKk0J5LtrMRMmNOVlPtcfIIWo Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Domain-Id: None Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Domain-Name: None Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Identity-Status: Confirmed Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Is-Admin-Project: True Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Openstack-Nova-Api-Version: 2.90 Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Domain-Id: default Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Domain-Name: Default Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Project-Name: admin Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Role: reader,admin,member Aug 24 09:57:01.858412 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Roles: reader,admin,member Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f71a167399734714a60b777eea4e2ecc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f71a167399734714a60b777eea4e2ecc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Tenant: admin Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Tenant-Id: f71a167399734714a60b777eea4e2ecc Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-Tenant-Name: admin Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User: admin Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Domain-Id: default Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Domain-Name: Default Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Id: 9757335c28f74ea5a8fbdedc0ae7682a Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: X-User-Name: admin Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: method: > Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: exception: Flavor 451 could not be found. Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: action_args: {'id': '451'} Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 09:57:01.860002 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:57:01.861127 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: ERROR nova.api.openstack.wsgi Aug 24 09:57:01.862770 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-d57da1f1-865e-484c-830a-32adc8d42cba admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105448) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 09:57:01.864006 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-d57da1f1-865e-484c-830a-32adc8d42cba admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012542 Aug 24 09:57:01.864006 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 27/52] 10.222.0.86 () {60 vars in 1193 bytes} [Wed Aug 24 09:57:01 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:02.823485 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:02.824520 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 Aug 24 09:57:02.824915 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 26/53] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 09:57:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:57:03.353962 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-de716078-db5e-4ad4-962f-7c8733be6622 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:57:03.372623 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-de716078-db5e-4ad4-962f-7c8733be6622 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.409662 Aug 24 09:57:03.373166 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 28/54] 10.222.0.86 () {64 vars in 1235 bytes} [Wed Aug 24 09:57:02 2022] POST /compute/v2.1/flavors => generated 404 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:57:04.329192 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:04.330191 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001423 Aug 24 09:57:04.330584 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 27/55] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 09:57:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:57:04.866803 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-eeee0585-b2d5-4f20-b499-609b7ac4aeb0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:57:04.885213 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-eeee0585-b2d5-4f20-b499-609b7ac4aeb0 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.414504 Aug 24 09:57:04.885753 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 29/56] 10.222.0.86 () {64 vars in 1235 bytes} [Wed Aug 24 09:57:04 2022] POST /compute/v2.1/flavors => generated 405 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:57:05.856386 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:05.857390 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001524 Aug 24 09:57:05.857785 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 28/57] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 09:57:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:57:06.393916 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-beca170a-08c9-4068-aee1-6c2f677b6027 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:57:06.412561 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-beca170a-08c9-4068-aee1-6c2f677b6027 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.415252 Aug 24 09:57:06.413098 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 30/58] 10.222.0.86 () {64 vars in 1235 bytes} [Wed Aug 24 09:57:05 2022] POST /compute/v2.1/flavors => generated 407 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 09:57:07.378921 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:07.379922 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001419 Aug 24 09:57:07.380359 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 29/59] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 09:57:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:57:07.898667 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-844f2509-08f1-45fc-b40b-c70f1cf33837 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 09:57:07.918306 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-844f2509-08f1-45fc-b40b-c70f1cf33837 admin admin] 10.222.0.86 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.400233 Aug 24 09:57:07.918886 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 31/60] 10.222.0.86 () {64 vars in 1235 bytes} [Wed Aug 24 09:57:07 2022] POST /compute/v2.1/flavors => generated 409 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 09:57:08.887962 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:08.889008 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001471 Aug 24 09:57:08.890055 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 30/61] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 09:57:08 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:57:09.411766 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-3c15a868-6fb9-48e4-9174-c768e384acf6 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:57:09.432108 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-3c15a868-6fb9-48e4-9174-c768e384acf6 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.406956 Aug 24 09:57:09.432628 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 32/62] 10.222.0.86 () {60 vars in 1202 bytes} [Wed Aug 24 09:57:09 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 09:58:02.553813 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:58:02.554891 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-65a5e76f-6b87-4cbe-9691-c4f214f68ce6 admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001520 Aug 24 09:58:02.555290 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 31/63] 10.222.0.86 () {54 vars in 931 bytes} [Wed Aug 24 09:58:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 09:58:02.566433 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.949318 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-16210199-f0b8-4f2b-83c6-aea9cddb3700 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:58:02.970359 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-16210199-f0b8-4f2b-83c6-aea9cddb3700 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.408361 Aug 24 09:58:02.970896 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 33/64] 10.222.0.86 () {60 vars in 1292 bytes} [Wed Aug 24 09:58:02 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 09:58:02.991884 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG dbcounter [-] [105447] Writing DB stats nova_api:SELECT=14 {{(pid=105447) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:02.995749 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-6aaeab55-4077-44f3-b3f1-d22835d84061 admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:58:03.003426 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-6aaeab55-4077-44f3-b3f1-d22835d84061 admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012971 Aug 24 09:58:03.003943 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 32/65] 10.222.0.86 () {60 vars in 1283 bytes} [Wed Aug 24 09:58:02 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:58:03.015842 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-aa4f4a15-9ed5-4cde-95ba-ecfb7f08f54f admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:58:03.023509 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-aa4f4a15-9ed5-4cde-95ba-ecfb7f08f54f admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011176 Aug 24 09:58:03.024029 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 34/66] 10.222.0.86 () {60 vars in 1286 bytes} [Wed Aug 24 09:58:03 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:58:03.035946 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:58:03.043492 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011114 Aug 24 09:58:03.044012 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 33/67] 10.222.0.86 () {60 vars in 1286 bytes} [Wed Aug 24 09:58:03 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 09:58:03.055306 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-21351465-3b95-4a17-b778-615dd54608ae admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 09:58:03.062790 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-21351465-3b95-4a17-b778-615dd54608ae admin admin] 10.222.0.86 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011038 Aug 24 09:58:03.063296 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 35/68] 10.222.0.86 () {60 vars in 1283 bytes} [Wed Aug 24 09:58:03 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:01:00.423393 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:01:00.424430 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001437 Aug 24 10:01:00.424825 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 34/69] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 10:01:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:01:00.566519 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_api:SELECT=6 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:00.725026 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-8a717692-5fa6-4cdd-bd45-17a17fec2d2a admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:01:00.727485 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:01:00.727485 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: warnings.warn(msg) Aug 24 10:01:00.728634 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8a717692-5fa6-4cdd-bd45-17a17fec2d2a admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:01:00.729106 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8a717692-5fa6-4cdd-bd45-17a17fec2d2a admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:01:00.729572 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8a717692-5fa6-4cdd-bd45-17a17fec2d2a admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:01:00.730281 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8a717692-5fa6-4cdd-bd45-17a17fec2d2a admin admin] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:01:00.730722 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8a717692-5fa6-4cdd-bd45-17a17fec2d2a admin admin] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:01:00.731133 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8a717692-5fa6-4cdd-bd45-17a17fec2d2a admin admin] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:01:00.765784 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-8a717692-5fa6-4cdd-bd45-17a17fec2d2a admin admin] 10.222.0.86 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.200754 Aug 24 10:01:00.766352 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 36/70] 10.222.0.86 () {60 vars in 1193 bytes} [Wed Aug 24 10:01:00 2022] GET /compute/v2.1/os-services => generated 791 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:01:01.605127 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:01:01.606125 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001409 Aug 24 10:01:01.606518 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 35/71] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 10:01:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:01:01.798752 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-2c72938f-6c5d-4373-90db-b51eea2ca66d admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:01:01.801215 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:01:01.801215 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: warnings.warn(msg) Aug 24 10:01:01.802317 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-2c72938f-6c5d-4373-90db-b51eea2ca66d admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:01:01.802792 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-2c72938f-6c5d-4373-90db-b51eea2ca66d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:01:01.803216 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-2c72938f-6c5d-4373-90db-b51eea2ca66d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:01:01.803958 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-2c72938f-6c5d-4373-90db-b51eea2ca66d admin admin] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:01:01.804426 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-2c72938f-6c5d-4373-90db-b51eea2ca66d admin admin] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:01:01.804839 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-2c72938f-6c5d-4373-90db-b51eea2ca66d admin admin] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:01:01.824336 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-2c72938f-6c5d-4373-90db-b51eea2ca66d admin admin] 10.222.0.86 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.080751 Aug 24 10:01:01.824882 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 37/72] 10.222.0.86 () {60 vars in 1193 bytes} [Wed Aug 24 10:01:01 2022] GET /compute/v2.1/os-services => generated 791 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:01:13.175945 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:01:13.177041 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001463 Aug 24 10:01:13.177408 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 36/73] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 10:01:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:01:13.317634 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_cell0:SELECT=4 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:13.318355 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_cell1:SELECT=4 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:13.318954 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG dbcounter [-] [105448] Writing DB stats nova_api:SELECT=8 {{(pid=105448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:13.367118 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-8c35c75a-84b6-49fb-a5c3-17ff8c897b1d admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:01:13.369475 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:01:13.369475 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: warnings.warn(msg) Aug 24 10:01:13.370557 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8c35c75a-84b6-49fb-a5c3-17ff8c897b1d admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:01:13.371020 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8c35c75a-84b6-49fb-a5c3-17ff8c897b1d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:01:13.371433 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8c35c75a-84b6-49fb-a5c3-17ff8c897b1d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:01:13.372192 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8c35c75a-84b6-49fb-a5c3-17ff8c897b1d admin admin] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:01:13.372635 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8c35c75a-84b6-49fb-a5c3-17ff8c897b1d admin admin] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:01:13.373041 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-8c35c75a-84b6-49fb-a5c3-17ff8c897b1d admin admin] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:01:13.397807 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-8c35c75a-84b6-49fb-a5c3-17ff8c897b1d admin admin] 10.222.0.86 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.084632 Aug 24 10:01:13.398284 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 38/74] 10.222.0.86 () {60 vars in 1193 bytes} [Wed Aug 24 10:01:13 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:01:14.221284 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: DEBUG nova.api.openstack.wsgi [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] Calling method '>' {{(pid=105447) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:01:14.222288 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: INFO nova.api.openstack.requestlog [None req-1e517dfb-7130-4a6f-95b5-fca46aa6551e admin admin] 10.222.0.86 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001423 Aug 24 10:01:14.222715 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: [pid: 105447|app: 0|req: 37/75] 10.222.0.86 () {56 vars in 1074 bytes} [Wed Aug 24 10:01:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:01:14.414636 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG nova.api.openstack.wsgi [None req-5581823d-e116-4827-bbb7-faf5d02ab5f8 admin admin] Calling method '>' {{(pid=105448) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:01:14.417209 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 10:01:14.417209 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: warnings.warn(msg) Aug 24 10:01:14.418302 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-5581823d-e116-4827-bbb7-faf5d02ab5f8 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:01:14.418769 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-5581823d-e116-4827-bbb7-faf5d02ab5f8 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:01:14.419197 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-5581823d-e116-4827-bbb7-faf5d02ab5f8 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:01:14.419898 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-5581823d-e116-4827-bbb7-faf5d02ab5f8 admin admin] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:01:14.420368 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-5581823d-e116-4827-bbb7-faf5d02ab5f8 admin admin] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:01:14.420780 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: DEBUG oslo_concurrency.lockutils [None req-5581823d-e116-4827-bbb7-faf5d02ab5f8 admin admin] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105448) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:01:14.442791 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: INFO nova.api.openstack.requestlog [None req-5581823d-e116-4827-bbb7-faf5d02ab5f8 admin admin] 10.222.0.86 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.078812 Aug 24 10:01:14.443267 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: [pid: 105448|app: 0|req: 39/76] 10.222.0.86 () {60 vars in 1193 bytes} [Wed Aug 24 10:01:14 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:02:13.514494 dv-cinder-854270-2-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Aug 24 10:02:13.515271 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: Wed Aug 24 10:02:13 2022 - graceful shutdown triggered... Aug 24 10:02:13.515714 dv-cinder-854270-2-iscsi devstack@n-api.service[105447]: Gracefully killing worker 1 (pid: 105447)... Aug 24 10:02:13.516167 dv-cinder-854270-2-iscsi devstack@n-api.service[105448]: Gracefully killing worker 2 (pid: 105448)... Aug 24 10:02:14.515886 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: worker 1 buried after 1 seconds Aug 24 10:02:14.516499 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: worker 2 buried after 1 seconds Aug 24 10:02:14.516499 dv-cinder-854270-2-iscsi devstack@n-api.service[105446]: goodbye to uWSGI. Aug 24 10:02:14.517536 dv-cinder-854270-2-iscsi systemd[1]: devstack@n-api.service: Succeeded. Aug 24 10:02:14.518096 dv-cinder-854270-2-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Aug 24 10:02:14.521540 dv-cinder-854270-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 24 10:02:14.531699 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 10:02:14 2022] *** Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: nodename: dv-cinder-854270-2-iscsi Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: machine: x86_64 Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: clock source: unix Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: pcre jit disabled Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: detected number of CPU cores: 4 Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: current working directory: / Aug 24 10:02:14.533881 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: detected binary path: /usr/bin/uwsgi-core Aug 24 10:02:14.534556 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: your processes number limit is 47680 Aug 24 10:02:14.534556 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: your memory page size is 4096 bytes Aug 24 10:02:14.534556 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: detected max file descriptor number: 2048 Aug 24 10:02:14.534556 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: lock engine: pthread robust mutexes Aug 24 10:02:14.534556 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: thunder lock: enabled Aug 24 10:02:14.534556 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 24 10:02:14.534556 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 10:02:14.573470 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: Python main interpreter initialized at 0x5578ef80f9c0 Aug 24 10:02:14.573470 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: python threads support enabled Aug 24 10:02:14.573470 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: your server socket listen backlog is limited to 100 connections Aug 24 10:02:14.573470 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: your mercy for graceful operations on workers is 90 seconds Aug 24 10:02:14.573858 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 10:02:14.573945 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: *** Operational MODE: preforking *** Aug 24 10:02:14.574189 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: *** uWSGI is running in multiple interpreter mode *** Aug 24 10:02:14.574189 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: spawned uWSGI master process (pid: 120642) Aug 24 10:02:14.574409 dv-cinder-854270-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 24 10:02:14.574966 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: spawned uWSGI worker 1 (pid: 120653, cores: 1) Aug 24 10:02:14.575315 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: spawned uWSGI worker 2 (pid: 120654, cores: 1) Aug 24 10:02:14.575476 dv-cinder-854270-2-iscsi devstack@n-api.service[120642]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 10:02:15.245004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 24 10:02:15.247172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 24 10:02:16.397445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 10:02:16.397445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: from cryptography.utils import int_from_bytes Aug 24 10:02:16.398064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 10:02:16.398064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: from cryptography.utils import int_from_bytes Aug 24 10:02:16.416252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 10:02:16.416252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: from cryptography.utils import int_from_bytes Aug 24 10:02:16.416883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 10:02:16.416883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: from cryptography.utils import int_from_bytes Aug 24 10:02:18.785082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 10:02:18.785975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 10:02:18.786680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 10:02:18.787214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 10:02:18.787635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 10:02:18.788221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.788729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.789220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.789710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.790386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.790877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.791214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.791550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.791891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.792188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.792518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.792854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-854270-2-iscsi {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.793188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.793506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.793830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.794230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.794475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.795005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.795323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.795847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.797227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.797227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.797442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.797697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.798026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.798346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.798848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.799341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.799672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.799987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.800600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-854270-2-iscsi {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.800974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.801299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.801624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.802057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.802376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.802694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.803024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.803353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.803671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.804014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.804358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.804684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.805010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.805316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.805630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.805946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.806270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.806579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.806888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.807143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.807454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.807778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.808135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.808460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.808876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.809304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.809798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.810292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.810659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.810978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.811309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.811610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.811922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.812260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.812573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.812887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.813211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.813531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.813858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.814175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.814498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.814913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.86 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.815168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.86 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.815480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.815827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.816172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.817033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.817356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.817682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.817995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.818345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.818664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.818995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.819334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.819654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.820914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.820914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.820914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.820914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.821579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.821579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.821916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.822069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.822649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.822649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.822971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.823258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.823565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.823874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.824209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.824522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.824835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.825158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.825483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.825800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.826109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.826415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.826725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.827048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.827358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.827668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.827991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.828331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.828658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.828960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.829266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.829575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.829893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.830195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.830501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.830820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.831127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.831434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.831749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.832868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.833453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.833836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.834330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.834695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.834918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.835272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.835523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 10:02:18.835998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.836248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 10:02:18.836381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.836381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.836645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 10:02:18.836790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.836922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 10:02:18.837061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.837194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 10:02:18.837323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.837467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.837606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.837750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.837887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.838025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.838156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.838303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.838452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.838765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.838916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.839158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.839293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.839435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.839589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.839811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.840035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.840035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.840445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.840445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.840706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.840847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.840978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.841108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-854270-2-iscsi {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.841252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.841383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.841520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.841659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.841798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.841939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.842086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.842219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.842356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.842488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.842628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.842770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.843001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.843372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.843589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.843589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.843849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.844010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.844186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.844315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.844458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.844588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.844726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.844934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.845070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.845200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.845329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.845466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.845612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.845743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.845883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.846070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.846214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.846390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.846539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.846704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.846837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.847020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.847162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-854270-2-iscsi {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.847345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.847488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.847718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.847853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.848056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.848206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.848385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.848529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.848705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.848892 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.849033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.849194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.849394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.849529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.849668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.849821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.850038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.850248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.850385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.850602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.850736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.850867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.850997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.851144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.851285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.851424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.851565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.851703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.851834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.851976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.852151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.852287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.852436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.852651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.852802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.853020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.853173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.853571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.853783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.853783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.854059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.854059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.854316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.854457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.854597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.854736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.854878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.855018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.855157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.855157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.855426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.855573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.855707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.855838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.855968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.856205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.856486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.856639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.856773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.857594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.857774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.857952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.858224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.858389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.858538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.858685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.858840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.859039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.859199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.859349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.859590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.859749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.859903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.860189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.860491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.860737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.860891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.861049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.861196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.861340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.861499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.861652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.861806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.861960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.86 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.862113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.862270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.86 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.862693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.862855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.863009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.863277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.863749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.863996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.864261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.864261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.864567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.864720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.864874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.865071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.865235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.865403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.865608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.865779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.865930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.866091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.866266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.866422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.866617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.866774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.866934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.867089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.867289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.867460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.867630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.867778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.867972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.868155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.868337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.868489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.868663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.868863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.869064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.869225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.869381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.869535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.869535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.869932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.870080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.870323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.870567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.870745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.870961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.871120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.871278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.871434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.871580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.871725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.871887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.872130 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.872130 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.872447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.872604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.872749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.872894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.873045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.873199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.873441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.873700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.874037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.874258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.874411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.874639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.874809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.874809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.875088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.875239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.875386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.875546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.875700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.875853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.876008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.876194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.876350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.876506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.876652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.876816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.876982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.877135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.877281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.877435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.877588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.877734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.877889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.878035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.878187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.878332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.878485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.878650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.879380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.879533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.879688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.879930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.880125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.880310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.880547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.880767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.880916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.881155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.881307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.881467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.881624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.881779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.881933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.882085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.882238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.882397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.882550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.882706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.882853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.883012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.883158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.883316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.883545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.883792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.883973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.884261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.884494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.884653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.884806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.884954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.885098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.885242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.885396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.885542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.885686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 10:02:18.885839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.885997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.886144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.886303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.886451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.886610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.886767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.886767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.887053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.887213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.887367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.887521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.887667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.887830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.887988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.888180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.888328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.888474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.888618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.888770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.888929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.889077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.889228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.889418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.889582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.889728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.889918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.890134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.890284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.890483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.890753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.890972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.891156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.891417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.891570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.891723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.891877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.892066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.892233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.892391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.892545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.892698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.892844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.893002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.893148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.893299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.893454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.893690 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.893919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.894080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.894080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.895125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.895291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.895447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.895602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.895758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.895911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.896106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.896259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.896406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.896567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.896725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.896878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.897030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.897193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.897347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.897509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.897655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.897807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.897961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.898113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.898278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.898438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.898632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.898784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.898970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.899121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.899277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.899480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.899648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.899803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.899949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.900131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.900306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.900453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.900609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.900781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.901027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.901256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.901434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.901670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.901821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.901965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.902109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.902261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.902422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.902568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.902720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.902866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.903018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.903163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.903163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.903455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.903618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.903849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.904124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.904288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.904464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.905001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.905001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.905454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.905617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.905773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.905921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.906066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.906226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.906380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.906526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.906760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.906923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.907070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.907224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.907379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.907525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.907688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.907834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.907979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.908158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.908333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.908481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.908626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.908778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.908931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.909087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.909240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.909393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.909591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.909742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.909896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.910091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.910251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.910416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.910563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.910708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.910853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.911025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.911265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.911265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.911657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.911873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.912032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.912215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.912371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.912524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.912688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.912835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.912988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.913143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.913288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.913440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.913593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.913746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.913987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.914195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.914347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.914500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.914656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.914810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.914970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.915205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.915366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.915600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.915760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.915916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.916082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.916233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.916380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.916524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.916687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.916857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.917005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.917157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.917303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.917456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.917602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.917757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.917911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.918056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.918207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.918353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.918508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.918653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.918815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.918961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.919106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.919251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.919405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.919559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.919754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.919912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.920113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.920275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.920432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.920577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.920732 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.920878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.921030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.921174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.921419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.921653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.921814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.922049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.922208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.922368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.922515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.922704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.922868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.923029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.923183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.923343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.923528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.923689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.923841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.924815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.924992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.925158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.925405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.925586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.925842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.926001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.926161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.926331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.926490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.926637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.926795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.926940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.926940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.927238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.927384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.927529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.927674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.927826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.927982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.928200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.928357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.928503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.928656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.928801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.928956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.929110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.929257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.929403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.929548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.929694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.929893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.930045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.930222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.930369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.930522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.930730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.930895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.931051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.931206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.931360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.931673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.931950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.932143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.932302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.932511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.932675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.932829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.932989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.933136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.933280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.933433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.933586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.933731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.933885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.934037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.934274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.934425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.934596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.934753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.934908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.935062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.935216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.935386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.935609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.935768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.935997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.936199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.936199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.936499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.936652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.936822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.936822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.937099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.937099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.937385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.937385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.937679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.937826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.937978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.938132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.938321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.938485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.938639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.938805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.939022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.939192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.939351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.939619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.939775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.940013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.940195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.940356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.940512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.940687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.940846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.941000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.941157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.941823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.942125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.942299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.942455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.942603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.942755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.942948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.943152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.943323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.943483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.943642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.943800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.943952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.944198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.944441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.944633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.944810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.944987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.945174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.945350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.945539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.945803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.946016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.946274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.946460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.946631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.946833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.946992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.947158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.947333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.947518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.947724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.947888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.948068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.948227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.948390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.948632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.948803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.948963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.949115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.949273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.949436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.949598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.949761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.949915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.950109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.950274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.950436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.950605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.950759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.950930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.951082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.951234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.951385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.951550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.951710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.951862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.952015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.952225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.952389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.952543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.952696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.952863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.953016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.953169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.953335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.953488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.953647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.953806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.953958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.954118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.954278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.954456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.954689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.954846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.955008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.955008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.955319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.955472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.955631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.955802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.956028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.956244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.956404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.956565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.956718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.956900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.957590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.957774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.957929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.958082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.958235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.958392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.958552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.958714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.958880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.959043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.959211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.959366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.959526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.959679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.959842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.959993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.960210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.960370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.960531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.960690 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.960844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.960995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.961167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.961319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.961473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.961473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.961782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.961936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.962098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.962258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.962417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.962578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.962737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.962899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.963059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.963219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.963371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.963531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.963691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.963869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.964024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.964308 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.964472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.964634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.964885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.965056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.965218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.965378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.965530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.965690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.965843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.966027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.966267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.966425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.966585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.966755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.966919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.967081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.967244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.967403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.967570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.967742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.967907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.968086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.968296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.968478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.968633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.968794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.969002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.969783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.969959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.970122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.970326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.970496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.970674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.970840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.970994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.971165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.971319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.971485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.971646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.971806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.971960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.972146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.972304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.972467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.972658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.972838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.973560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.973903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.974070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.974234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.974388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.974549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.974701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.974879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.975120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.975279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.975440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.975601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.975766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.975934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.976241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.976493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.976653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.976805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.976957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.977121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.977281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.977441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.977605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.977757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.977916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.978079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.978240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.978405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.978557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.978716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.978876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.979038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.979193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.979346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.979497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.979656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.979887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.980062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.980232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.980446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.980612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.980776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.980939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.981098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.981259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.981413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.981564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.981716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.981876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.982029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.982181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.982332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.982497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.982649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.982811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.982964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.983125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.983286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.983447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.983609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.983770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.983933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.984117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.984277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.984429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.984603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.984764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.984924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.985096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.985329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.985487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.985649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.985803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.985803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.986093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.986253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.986406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.986662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.986841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.986995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.987147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.987300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.987728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.987883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.988060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.988248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.988412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.988574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.988748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.988911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.989074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.989074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.989363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.989363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.989683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.989852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.990006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.990158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.990323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.990517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.990674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.990836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.990997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.991161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.991328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.991490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.991643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.991795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.991953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.992169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.992345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.992345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.992636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.992636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.992942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.993103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.993280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.993280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.993578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.993578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.993867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.994030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.994190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.994344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.994496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.994664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.994825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.994988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.995141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.995323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.995555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.995717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.995877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.996067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.996228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.996381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.996544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.996821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.997010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.997174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.997378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.997547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.997718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.997879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.998041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.998198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.998350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.998503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.998662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.998816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.998984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.998984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.999294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.999485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.999644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.999797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.999970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:18.999970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.000335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.000335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.000686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.000850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.001005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.001164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.001318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.001483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.001636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.001797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.001951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.002104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.002256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.002416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.002652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.002815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.002976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.003129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.003281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.003441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.003619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.004317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.004499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.004653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.004812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.004971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.005138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.005299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.005453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.005698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.005857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.006009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.006184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.006339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.006500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.006654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.006833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.007081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.007246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.007246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.007547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.007699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.007851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.008018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.008223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.008393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.008554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.008716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.008869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.009035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.009189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.009341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.009492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.009653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.009806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.009972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.010125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.010277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.010439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.010644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.010814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.010975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.011135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.011296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.011456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.012163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.012343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.012343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.012638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.012792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.012952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.013117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.013280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.013440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.013593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.013593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.013899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.014068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.014222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.014383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.014535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.014694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.014854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.015032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.015186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.015346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.015506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.015677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.015906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.016116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.016277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.016430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.016582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.016745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.016906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.017088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.017088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.017484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.017638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.017798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.017971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.018142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.018142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.018436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.018587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.018747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.018912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.019067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.019226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.019554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.020246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.020412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.020580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.020828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.021003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.021159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.021319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.021471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.021633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.021795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.021948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.022120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.022281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.022449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.022611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.022772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.022932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.023092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.023253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.023421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.023575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.023728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.023879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.024038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.024272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.024443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.024596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.024755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.024908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.025068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.025228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.025397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.025558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.025712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.025879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.026108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.026266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.026266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.026579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.026732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.026885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.026885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.027190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.027361 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.027590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.027764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.027926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.028175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.028351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.028351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.028655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.028815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.028977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.029130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.029297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.029458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.029620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.029620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.029909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.030076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.030237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.030398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.030569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.030732 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.030929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.031098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.031260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.031260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.031557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.031712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.031874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.032075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.032239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.032404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.032566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.032738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.032899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.033071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.033245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.033400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.033559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.033720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.033882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.034037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.034189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.034354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.034515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.034676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.034922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.035116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.035273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.035425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.035585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.035739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.035903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.036088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.036330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.036489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.036641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.036792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.036971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.037126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.037286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.037447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.037626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.037872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.038040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.038201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.038363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.038525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.038678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.038849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.039011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.039184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.039345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.039505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.039658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.039825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.039979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.040200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.044174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.044378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.044555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.044715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.044870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.045024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.045201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.045357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.045524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.045687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.045841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.046002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.046164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.046339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.046508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.046662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.046822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.046975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.047147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.047308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.047308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.047622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.047776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.047949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.048176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.048353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.048517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.048678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.048833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.048992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.049166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.049337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.049492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.049652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.049812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.049973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.050134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.050297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.050459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.050613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.050800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.050954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.051116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.051280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.051449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.051604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.051758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.051921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.052107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.052275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.052430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.052583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.052743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.052905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.053072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.053244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.053399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.053561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.053716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.053879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.054043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.054246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.054404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.054565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.054728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.054895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.055092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.055092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.055455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.055650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.055807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.055969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.056160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.056315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.056478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.056634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.056797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.056952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.057104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.057256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.057419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.057580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.057743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.057906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.058059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.058219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.058383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.058536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.058698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.058853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.059016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.059181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.059345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.059520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.059675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.059827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.059987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.061971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.062147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.062315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.062315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.062619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.062772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.062924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.063076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.063252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.063406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.063558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.063717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.063871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.064024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.064276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.064445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.064606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.064759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.064911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.065076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.065239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.065411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.065573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.065734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.065887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.066049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.066218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.066911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.067066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.067225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.067394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.067562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.067716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.067880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.068032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.068210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.068394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.068549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.068701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.068860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.068860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.069159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.069321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.069482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.069642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.069803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.069967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.070119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.070315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.070476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.070636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.070798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.070951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.071111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.071354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.071523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.071678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.071830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.071994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.072182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.072338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.072491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.072644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.072797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.072976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.073147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.073301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.073455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.073623 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.073776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.073935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.074109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.074262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.074464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.074622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.074775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.074942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.075095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.075247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.075400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.075552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.075705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.076190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.076366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.076521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.076682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.076844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.076998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.077164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 10:02:19.077320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.077320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.077320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.077320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.077320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.077320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.077320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.077320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.078975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.078975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.078975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.078975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.078975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.078975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.078975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.078975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.080180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.080180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.080180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.080180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.080180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.080180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.080180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.080180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.081432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.081432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.081432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.081432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.081432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.081432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.081432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.081432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.083348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.083348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.083348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.083348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.083348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.083348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.083348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.083348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.084585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.084585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.084585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.084585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.084585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.084585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.084585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.084585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.086850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.086850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.086850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.086850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.086850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.086850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.086850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.086850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.088010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.088010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.088010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.088010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.088010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.088010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.088010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.088010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.089269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.089269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.089269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.089269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.089269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.089269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.089269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.089269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.090439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.090439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.090439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.090439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.090439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.090439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.090439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.090439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.091660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.091660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.091660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.091660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.091660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.091660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.091660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.091660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.093090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.093090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.093090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.093090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.093090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.093090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.093090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.093090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.094270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.094270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.094270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.094270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.094270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.094270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.094270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.094270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.095489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.095489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.095489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.095489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.095489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.095489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.095489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.095489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.096706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.096706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.096706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.096706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.096706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.096706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.096706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.096706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.097930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.097930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.097930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.097930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.098505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.098505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.098812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.098964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.099206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.099525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.099847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.100195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.100522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.100841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.101311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.101646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.102010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.102345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.102665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.103040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.103326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.103680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.104038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.104442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.104778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.105100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.105447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.105784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 10:02:19.106061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 10:02:19.116656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO dbcounter [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Registered counter for database nova_api Aug 24 10:02:19.138322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_db.sqlalchemy.engines [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120653) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:02:19.145843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writer thread running {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:02:19.171874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO dbcounter [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Registered counter for database nova_api Aug 24 10:02:19.193268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_db.sqlalchemy.engines [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:02:19.200116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writer thread running {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:02:19.202132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.context [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),62974e32-dd59-452e-afe9-bdb8e3ca6ae3(cell1) {{(pid=120653) load_cells /opt/stack/nova/nova/context.py:464}} Aug 24 10:02:19.206382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:02:19.207372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:02:19.209097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:02:19.209860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:02:19.210333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:02:19.212432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:02:19.214501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO dbcounter [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Registered counter for database nova_cell0 Aug 24 10:02:19.218760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO dbcounter [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Registered counter for database nova_cell1 Aug 24 10:02:19.226640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_db.sqlalchemy.engines [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120653) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:02:19.228373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_db.sqlalchemy.engines [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120653) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:02:19.233988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writer thread running {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:02:19.236577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writer thread running {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:02:19.257463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.context [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),62974e32-dd59-452e-afe9-bdb8e3ca6ae3(cell1) {{(pid=120654) load_cells /opt/stack/nova/nova/context.py:464}} Aug 24 10:02:19.261561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:02:19.262040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:02:19.263644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:02:19.264619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:02:19.265078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:02:19.267094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:02:19.269130 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO dbcounter [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Registered counter for database nova_cell0 Aug 24 10:02:19.273758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO dbcounter [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Registered counter for database nova_cell1 Aug 24 10:02:19.280917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_db.sqlalchemy.engines [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:02:19.283588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_db.sqlalchemy.engines [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:02:19.288563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writer thread running {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:02:19.294305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writer thread running {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:02:19.382693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO dbcounter [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Registered counter for database nova_cell0 Aug 24 10:02:19.389603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_db.sqlalchemy.engines [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120653) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:02:19.394026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writer thread running {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:02:19.436878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO dbcounter [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Registered counter for database nova_cell0 Aug 24 10:02:19.444068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_db.sqlalchemy.engines [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 10:02:19.448531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writer thread running {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 10:02:19.711637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: WARNING keystonemiddleware.auth_token [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 24 10:02:19.719000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: WARNING keystonemiddleware.auth_token [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 24 10:02:19.772269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: WARNING keystonemiddleware.auth_token [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 24 10:02:19.781785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: WARNING keystonemiddleware.auth_token [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 24 10:02:20.066932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: WARNING keystonemiddleware.auth_token [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 24 10:02:20.071269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: WARNING keystonemiddleware.auth_token [None req-d029dc49-8c81-40b5-851b-e34b7a46fd87 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 24 10:02:20.076695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5578ef80f9c0 pid: 120653 (default app) Aug 24 10:02:20.142768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: WARNING keystonemiddleware.auth_token [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 24 10:02:20.149134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: WARNING keystonemiddleware.auth_token [None req-137c794b-a755-4d22-8d2c-5908e3622893 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 24 10:02:20.154739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5578ef80f9c0 pid: 120654 (default app) Aug 24 10:02:55.904465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.909422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.910447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.911871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.058737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.060796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.061693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.062731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:57.562335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-34b03306-fde4-4932-ae26-871037141978 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Action: 'create', calling method: .version_select at 0x7fa33f94a700>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1239292542", "metadata": {"Type": "work"}}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:02:57.645117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-be461391-48b8-4828-9744-fa248d4176a5 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Action: 'create', calling method: .version_select at 0x7fa33f9485e0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-64570990"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:02:59.385279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-34b03306-fde4-4932-ae26-871037141978 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 3.493569 Aug 24 10:02:59.396237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1/1] 10.222.0.86 () {60 vars in 1156 bytes} [Wed Aug 24 10:02:55 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 3506 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 24 10:02:59.402054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-82e9474d-a3c7-4cf4-ac5f-ab8c52dbf543 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f81c550>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:02:59.539307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-be461391-48b8-4828-9744-fa248d4176a5 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "POST /compute/v2.1/os-volumes" status: 200 len: 343 microversion: 2.1 time: 3.489599 Aug 24 10:02:59.541582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1/2] 10.222.0.86 () {60 vars in 1155 bytes} [Wed Aug 24 10:02:56 2022] POST /compute/v2.1/os-volumes => generated 343 bytes in 3494 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 24 10:02:59.567166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-80757fa2-4798-475b-86ee-a6724f7bd94f tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f81a4c0>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:02:59.638223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-82e9474d-a3c7-4cf4-ac5f-ab8c52dbf543 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.240306 Aug 24 10:02:59.638815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 2/3] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:02:59 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:02:59.781889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-80757fa2-4798-475b-86ee-a6724f7bd94f tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3" status: 200 len: 343 microversion: 2.1 time: 0.221012 Aug 24 10:02:59.790578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 2/4] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:02:59 2022] GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 => generated 343 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:00.654215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e863dab2-04b4-4b23-a595-c991b1bdcf8a tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f81cc10>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:00.805722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e863dab2-04b4-4b23-a595-c991b1bdcf8a tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 354 microversion: 2.1 time: 0.156704 Aug 24 10:03:00.811693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 3/5] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:00 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 354 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:00.815455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-339451cb-036c-4a66-9193-c26a27b8e72c tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f81ac10>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:00.824323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a3c7f3dd-f55f-4350-be22-0fc05891c185 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f837310>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:00.952907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-339451cb-036c-4a66-9193-c26a27b8e72c tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3" status: 200 len: 344 microversion: 2.1 time: 0.154693 Aug 24 10:03:00.952907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 3/6] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:00 2022] GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:00.969626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e1eb82af-7a3f-4772-9ef8-c2a89a536e45 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Action: 'create', calling method: .version_select at 0x7fa33f831280>, body: {"snapshot": {"volume_id": "67c1c0ca-8964-4e1a-b566-699c6e7456c3", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-2075637027"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:01.108205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a3c7f3dd-f55f-4350-be22-0fc05891c185 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 354 microversion: 2.1 time: 0.287928 Aug 24 10:03:01.108761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 4/7] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:00 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 354 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:01.124822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fbcf300a-0b9c-49d0-80c1-eda21dc74097 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f81cc10>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:01.439783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e1eb82af-7a3f-4772-9ef8-c2a89a536e45 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.472067 Aug 24 10:03:01.440970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 4/8] 10.222.0.86 () {60 vars in 1162 bytes} [Wed Aug 24 10:03:00 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:01.456922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bab0a9b6-eec7-4927-a9e7-ce15ff49e79c tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f831d30>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:01.587559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fbcf300a-0b9c-49d0-80c1-eda21dc74097 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "DELETE /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 202 len: 0 microversion: 2.1 time: 0.467473 Aug 24 10:03:01.588193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 5/9] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:03:01 2022] DELETE /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 0 bytes in 469 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 24 10:03:01.601689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-03747d87-fc88-4ae5-beb3-9205cf961736 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f81c550>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:01.620601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bab0a9b6-eec7-4927-a9e7-ce15ff49e79c tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.173131 Aug 24 10:03:01.621608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 5/10] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:01 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:01.633961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-36e94416-1455-425b-9b90-86b559b18c24 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f831550>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:01.701174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-03747d87-fc88-4ae5-beb3-9205cf961736 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.103884 Aug 24 10:03:01.701739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 6/11] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:01 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:01.755054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-36e94416-1455-425b-9b90-86b559b18c24 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.125211 Aug 24 10:03:01.755676 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 6/12] 10.222.0.86 () {58 vars in 1140 bytes} [Wed Aug 24 10:03:01 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:01.766859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0469a920-615f-4111-a201-57b209197b69 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f8374c0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:01.869228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0469a920-615f-4111-a201-57b209197b69 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.106015 Aug 24 10:03:01.870036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 7/13] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:01 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:02.716624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-72434388-8c17-437f-be2e-7b4ae740c0f9 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f948ee0>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:02.832537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-72434388-8c17-437f-be2e-7b4ae740c0f9 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.120260 Aug 24 10:03:02.833224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 7/14] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:02 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:02.881675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cbf57260-66e8-44a1-a789-dd0f6e39ee92 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f81c310>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:02.931369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cbf57260-66e8-44a1-a789-dd0f6e39ee92 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.053410 Aug 24 10:03:02.931944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 8/15] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:02 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:03.852982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a344f53f-91c7-45b3-b33d-4e1a35448400 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f81a280>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:03.944633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-84c03fd0-5257-40db-b9ae-d62ed5040ff0 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f81c940>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:03.970385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a344f53f-91c7-45b3-b33d-4e1a35448400 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.127200 Aug 24 10:03:03.976957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 8/16] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:03 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:03.996186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-84c03fd0-5257-40db-b9ae-d62ed5040ff0 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.055327 Aug 24 10:03:03.997536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 9/17] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:03 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:04.983424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-98bb099c-80de-4034-a47c-48ad90dc914c tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f81aaf0>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:05.009608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f901c1bc-dd0e-4d67-b6a2-b56cc8c93fd1 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f837280>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:05.112855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-98bb099c-80de-4034-a47c-48ad90dc914c tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.133295 Aug 24 10:03:05.113657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 9/18] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:04 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:05.171743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f901c1bc-dd0e-4d67-b6a2-b56cc8c93fd1 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.165778 Aug 24 10:03:05.172660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 10/19] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:05 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:06.129540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e2262207-113f-4952-a69e-7f759157e694 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f831430>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:06.191362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6621dcac-7a06-4706-8f2d-08568af1406d tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f81cc10>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:06.243382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e2262207-113f-4952-a69e-7f759157e694 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.120061 Aug 24 10:03:06.251112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 10/20] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:06 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:06.283800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6621dcac-7a06-4706-8f2d-08568af1406d tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.096810 Aug 24 10:03:06.284843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 11/21] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:06 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:06.728897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-923aa403-466b-46e1-920a-e072843e8553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Action: 'create', calling method: .version_select at 0x7fa33f831ca0>, body: {"keypair": {"name": "tempest-keypair-1318750226", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOi7Cnr0XZr/o8u+qhGk6T/h+6S7amzUJiqkEI5fYdHZWXYfa46QVrcPEWqv2usosSkP6jFR3adDivjpfF/70p9szeR0JRm1qSOe3g9Yy79Vbjv3pyAKsZLX3jnhGmWyHg=="}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:06.844424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-923aa403-466b-46e1-920a-e072843e8553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Getting quotas for project 7c1ad7a7198d4e78b9d03f00ce65e091. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:03:06.856790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-923aa403-466b-46e1-920a-e072843e8553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Getting quotas for user df9b402589fa4020bc241a4b9c06dd91 and project 7c1ad7a7198d4e78b9d03f00ce65e091. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:03:06.991731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-923aa403-466b-46e1-920a-e072843e8553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.738646 Aug 24 10:03:06.994713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 11/22] 10.222.0.86 () {62 vars in 1200 bytes} [Wed Aug 24 10:03:06 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 742 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 24 10:03:07.258713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-da409e7c-4aa3-47ca-8422-9b30548abae3 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f81c820>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:07.297118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-089fae78-4824-4b23-b9a2-3e70f71a98df tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f79f5e0>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:07.438279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-da409e7c-4aa3-47ca-8422-9b30548abae3 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.183547 Aug 24 10:03:07.448623 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 12/23] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:07 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:07.464316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-089fae78-4824-4b23-b9a2-3e70f71a98df tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.171803 Aug 24 10:03:07.464960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 12/24] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:07 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:08.453585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-be4763b3-94c0-49d5-8d92-bc114392d369 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f8379d0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:08.490140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-00d1927f-bdf4-4e0e-a32e-ff83e34d453a tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f79fca0>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:08.643181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-be4763b3-94c0-49d5-8d92-bc114392d369 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.193490 Aug 24 10:03:08.643790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 13/25] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:08 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:08.652226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-00d1927f-bdf4-4e0e-a32e-ff83e34d453a tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 288 microversion: 2.1 time: 0.177492 Aug 24 10:03:08.652896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 13/26] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:08 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 288 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:08.665125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c5b48ff2-768e-478b-9160-20fa012ec927 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f837af0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:08.792891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c5b48ff2-768e-478b-9160-20fa012ec927 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "DELETE /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 202 len: 0 microversion: 2.1 time: 0.131534 Aug 24 10:03:08.797245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 14/27] 10.222.0.86 () {58 vars in 1254 bytes} [Wed Aug 24 10:03:08 2022] DELETE /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 0 bytes in 136 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:03:08.802784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-96af30f0-b34f-42a3-b90c-8b7681b89e5f tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f81c3a0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:08.897782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-96af30f0-b34f-42a3-b90c-8b7681b89e5f tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.099346 Aug 24 10:03:08.903582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 15/28] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:08 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:09.027213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1145650925", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:09.196627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] validate_networks() for [('3742113f-8471-4780-be0c-20ea54885f07', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:03:09.656748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-669710d5-eace-4d41-b137-adcb044ffcea tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33f81c9d0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:09.754559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-669710d5-eace-4d41-b137-adcb044ffcea tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.101456 Aug 24 10:03:09.755122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 16/29] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:09 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:09.910071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-48bf3214-23f1-4229-b4da-bc6b9f638449 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33f837d30>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:09.971118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-48bf3214-23f1-4229-b4da-bc6b9f638449 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.065432 Aug 24 10:03:09.977158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 17/30] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:09 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:10.076435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1713477433", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "26b92eb1-90dd-4030-8ddb-d65fcb93c009"}], "security_groups": [{"name": "tempest-securitygroup--1285666544"}], "key_name": "tempest-keypair-1318750226", "user_data": "}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:10.314700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] validate_networks() for [('26b92eb1-90dd-4030-8ddb-d65fcb93c009', None, None, None, None, None)] {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:03:10.375405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:03:10.377104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:03:10.377798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:03:10.378406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:03:10.379027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:03:10.406631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:10.409446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:10.409937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:10.412567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:10.421019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:10.421550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:10.458385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Getting quotas for project 1bdfd8ab42f9496d87a8cfcf78d7edc3. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:03:10.467974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Getting quotas for user 46c01f8b99e24be5b6c2d73b9b1bd9b8 and project 1bdfd8ab42f9496d87a8cfcf78d7edc3. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:03:10.486754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:03:10.493633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:03:10.609704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d7717186-5cb0-447a-927b-7d3e8cb1aed7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.874271 Aug 24 10:03:10.610467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 14/31] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:03:08 2022] POST /compute/v2.1/servers => generated 383 bytes in 1876 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:03:10.813479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1108756248", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "networks": [{"uuid": "825b50d1-3ec1-448f-9cf8-4790cf896524"}]}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:10.941559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] validate_networks() for [('825b50d1-3ec1-448f-9cf8-4790cf896524', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:03:11.548950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:03:11.549553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:03:11.549995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:03:11.550592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:03:11.551176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:03:11.573872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:11.574402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:11.574865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:11.576411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:11.576878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:11.578148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:11.669176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Getting quotas for project 7c1ad7a7198d4e78b9d03f00ce65e091. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:03:11.685051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Getting quotas for user df9b402589fa4020bc241a4b9c06dd91 and project 7c1ad7a7198d4e78b9d03f00ce65e091. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:03:11.722810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:03:11.754007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:03:11.781106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:03:11.781609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:03:11.782054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:03:11.782570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:03:11.782954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:03:11.795893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:11.796418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:11.798625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:11.799407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:11.799872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:11.801685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:11.878689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Getting quotas for project 8d455181e7e046f480437a8ab3620746. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:03:11.901273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Getting quotas for user c4b8d531e382487a938546c009398b2e and project 8d455181e7e046f480437a8ab3620746. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:03:11.961341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:03:11.973407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:03:12.043285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-49b37ab8-1889-4e65-8d43-75481416ec16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.973411 Aug 24 10:03:12.047269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 18/32] 10.222.0.86 () {62 vars in 1188 bytes} [Wed Aug 24 10:03:10 2022] POST /compute/v2.1/servers => generated 402 bytes in 1975 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 24 10:03:12.057028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5b1e0e85-1fe0-4182-811d-63d4d7cd38f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:12.058577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5b1e0e85-1fe0-4182-811d-63d4d7cd38f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:12.098356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8ed3f9e5-bab6-45d0-879c-ee394af095fd tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.483410 Aug 24 10:03:12.104831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 15/33] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:03:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 1493 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:03:12.125480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-07cc6991-c6d5-47bb-8bba-7f72d238a002 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33eee9700>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:12.297270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-07cc6991-c6d5-47bb-8bba-7f72d238a002 tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.190119 Aug 24 10:03:12.297903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 16/34] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:12 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:12.304704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9b7395b7-119c-42f1-b069-7a66068a44ab tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33eef2790>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:12.353483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5b1e0e85-1fe0-4182-811d-63d4d7cd38f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:12.388073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5b1e0e85-1fe0-4182-811d-63d4d7cd38f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1222 microversion: 2.1 time: 0.338828 Aug 24 10:03:12.388838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 19/35] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:12 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1222 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:12.400302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9b7395b7-119c-42f1-b069-7a66068a44ab tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.093026 Aug 24 10:03:12.400302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 17/36] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:12 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:12.413694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8385ad1d-8814-4985-8f88-12a0da814825 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:12.413694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8385ad1d-8814-4985-8f88-12a0da814825 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:12.455681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d7917533-a1f8-46b1-a718-3f65bdf2d143 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:12.456532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d7917533-a1f8-46b1-a718-3f65bdf2d143 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:12.584534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d7917533-a1f8-46b1-a718-3f65bdf2d143 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:12.592855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d7917533-a1f8-46b1-a718-3f65bdf2d143 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1215 microversion: 2.1 time: 0.165816 Aug 24 10:03:12.599402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 20/37] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:12 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1215 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:12.614090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1322121180", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:12.811677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] validate_networks() for [('3742113f-8471-4780-be0c-20ea54885f07', None, None, None, None, None)] {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:03:12.838131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8385ad1d-8814-4985-8f88-12a0da814825 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:12.872631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8385ad1d-8814-4985-8f88-12a0da814825 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:12.874774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8385ad1d-8814-4985-8f88-12a0da814825 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:12.882084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8385ad1d-8814-4985-8f88-12a0da814825 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1282 microversion: 2.20 time: 0.483672 Aug 24 10:03:12.883045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 18/38] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:12 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1282 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:13.315883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aec8d80d-48bc-4d70-bed5-5a329db2763f tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33de9ae50>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:13.473871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aec8d80d-48bc-4d70-bed5-5a329db2763f tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 200 len: 353 microversion: 2.1 time: 0.165517 Aug 24 10:03:13.474452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 19/39] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:13 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 353 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:13.480440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-51770ff5-ee89-4bb1-aa38-f78f516d2fc2 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33de9f550>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:13.570300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-51770ff5-ee89-4bb1-aa38-f78f516d2fc2 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 200 len: 287 microversion: 2.1 time: 0.093814 Aug 24 10:03:13.571079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 20/40] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:13 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 287 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:13.621101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:13.622188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:13.635523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:13.636659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:13.637366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:13.638522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:13.639246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:13.640373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:13.823166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:13.838845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3bd1e868-0713-4e46-a495-fec268832425 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1211 microversion: 2.1 time: 0.224145 Aug 24 10:03:13.853858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 21/41] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:13 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1211 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:13.909273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:13.910530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:13.923054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:13.926740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:13.927578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:13.929080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:13.930324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:13.932154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:14.011047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:03:14.012267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:03:14.012932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:03:14.013478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:03:14.014038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:03:14.039165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:14.044384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:14.044758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:14.047025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:14.050844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:14.051443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:14.051896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:14.058315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:14.060248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:14.061493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9c47b46b-7407-4315-a18c-6650735a312d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1286 microversion: 2.20 time: 0.158372 Aug 24 10:03:14.076668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 22/42] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1286 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:03:14.098331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Getting quotas for project 1bdfd8ab42f9496d87a8cfcf78d7edc3. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:03:14.103177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Getting quotas for user 46c01f8b99e24be5b6c2d73b9b1bd9b8 and project 1bdfd8ab42f9496d87a8cfcf78d7edc3. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:03:14.119342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:03:14.134415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:03:14.211608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c3a1cdc3-c3c5-482e-9b35-5c67f7f5e012 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.610378 Aug 24 10:03:14.213990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 21/43] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:03:12 2022] POST /compute/v2.1/servers => generated 383 bytes in 1613 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:03:14.223658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-196191ec-776e-42b3-9e56-d7a756dbbc8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:14.224655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-196191ec-776e-42b3-9e56-d7a756dbbc8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:14.304776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-196191ec-776e-42b3-9e56-d7a756dbbc8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:14.313154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-196191ec-776e-42b3-9e56-d7a756dbbc8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1222 microversion: 2.1 time: 0.093755 Aug 24 10:03:14.328742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 23/44] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:14 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1222 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:14.488385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b7db80ec-78d3-49e0-824e-cb588bc628fe tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Calling method '.version_select at 0x7fa33e68d790>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:14.549244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-b7db80ec-78d3-49e0-824e-cb588bc628fe tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] HTTP exception thrown: Volume 5e857bc0-cbc5-40b1-9060-d147bcdcfd82 could not be found. Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b7db80ec-78d3-49e0-824e-cb588bc628fe tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Request method failure captured: Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 HTTP/1.1 Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdPSfvHdPTE0WNXUZtncjnrDIOeImOzcchDEzdZfDYdwjHyEdSZwKFHjp4Ga2BkmTijoWbsNjP2J6ULU6kgXWUAxf5XWdGaB3RezrRGFIXegGCa1Xr5Z6itf-G76KLo8929jXMlig2PsvgjM4kda4DSfHxOkEwo62xiyAgR75bGLPg Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 80f07909b541427f857887bfe0426de5 Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesGetTestJSON-1620581417 Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:03:14.554303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/80f07909b541427f857887bfe0426de5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/80f07909b541427f857887bfe0426de5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_80f07909b541427f857887bfe0426de5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/80f07909b541427f857887bfe0426de5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesGetTestJSON-1620581417 Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 80f07909b541427f857887bfe0426de5 Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesGetTestJSON-1620581417 Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesGetTestJSON-1620581417-project Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 1b1e2cf23d2f4167aec1221848a5f832 Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesGetTestJSON-1620581417-project Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Volume 5e857bc0-cbc5-40b1-9060-d147bcdcfd82 could not be found. Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '5e857bc0-cbc5-40b1-9060-d147bcdcfd82'} Aug 24 10:03:14.555524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 5e857bc0-cbc5-40b1-9060-d147bcdcfd82 could not be found. (HTTP 404) (Request-ID: req-e190724f-c30d-4679-bc16-429283ed80fc) Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:14.556280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:14.557299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 5e857bc0-cbc5-40b1-9060-d147bcdcfd82 could not be found. Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 24 10:03:14.558303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 5e857bc0-cbc5-40b1-9060-d147bcdcfd82 could not be found. Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 5e857bc0-cbc5-40b1-9060-d147bcdcfd82 could not be found. Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b7db80ec-78d3-49e0-824e-cb588bc628fe tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] Returning 404 to user: Volume 5e857bc0-cbc5-40b1-9060-d147bcdcfd82 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b7db80ec-78d3-49e0-824e-cb588bc628fe tempest-VolumesGetTestJSON-1620581417 tempest-VolumesGetTestJSON-1620581417-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82" status: 404 len: 109 microversion: 2.1 time: 0.073517 Aug 24 10:03:14.561231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 22/45] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:14 2022] GET /compute/v2.1/os-volumes/5e857bc0-cbc5-40b1-9060-d147bcdcfd82 => generated 109 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:03:14.587113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-628c7d2b-fdbe-4da8-acab-f61a546d420f tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33ddccaf0>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:14.638550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-628c7d2b-fdbe-4da8-acab-f61a546d420f tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] HTTP exception thrown: Snapshot ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b could not be found. Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-628c7d2b-fdbe-4da8-acab-f61a546d420f tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Request method failure captured: Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b HTTP/1.1 Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdPDX7uTY0d2sh37QEWMwf0NdqDYMaHx9EfWmaeFZm5wIcF-xbNOXXJx656u_-6yzoMUW1hulU9pQP3UT1fF0ktCjyLK3AIaAkRDqv8TQKaQ_6UDFKxIJzDxwnJ7Fe3ztgkOYLJRA8wJXo2n-i4zWuH3HF4sYmSlnm_XW_BD19fND0 Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 9e7aa26bd9e64b30a6c0c3d3824a1dda Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:03:14.645280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 9e7aa26bd9e64b30a6c0c3d3824a1dda Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesSnapshotsTestJSON-1940705297-project Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 2e5d00edb03c4b6c91290f1d4bf4c28c Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1940705297-project Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Snapshot ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b could not be found. Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': 'ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b'} Aug 24 10:03:14.646508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b could not be found. (HTTP 404) (Request-ID: req-5a8d4cf8-5fa3-469a-a189-f093f691676a) Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.647968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:14.649364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b could not be found. Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Aug 24 10:03:14.651728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b could not be found. Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b could not be found. Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-628c7d2b-fdbe-4da8-acab-f61a546d420f tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Returning 404 to user: Snapshot ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:14.653221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-628c7d2b-fdbe-4da8-acab-f61a546d420f tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b" status: 404 len: 111 microversion: 2.1 time: 0.066421 Aug 24 10:03:14.655276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 24/46] 10.222.0.86 () {58 vars in 1251 bytes} [Wed Aug 24 10:03:14 2022] GET /compute/v2.1/os-snapshots/ab72fb9d-d4f4-4860-9a1c-2cc0c3dadf0b => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:03:14.677116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-997cb41c-f2a5-4a5b-a1f9-7d3b09ce3471 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa3401f3b80>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:14.843379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-997cb41c-f2a5-4a5b-a1f9-7d3b09ce3471 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "DELETE /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3" status: 202 len: 0 microversion: 2.1 time: 0.190988 Aug 24 10:03:14.847336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 23/47] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:03:14 2022] DELETE /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 => generated 0 bytes in 196 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:03:14.854578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1d3e42ca-4665-442e-ba6e-a8038d04ea17 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33ddcc940>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:14.866037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:14.866649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:14.882651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:14.883195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:14.884682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:14.886465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:14.886946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:14.889525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:14.945319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1d3e42ca-4665-442e-ba6e-a8038d04ea17 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3" status: 200 len: 343 microversion: 2.1 time: 0.096745 Aug 24 10:03:14.946080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 25/48] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:14 2022] GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 => generated 343 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:15.090087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:15.091466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:15.108967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:15.112325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:15.113990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:15.117014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:15.118972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:15.120312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:15.231950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:15.267045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-79dcb98b-f51f-4eb3-ae0e-ddd81a1183bf tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1275 microversion: 2.1 time: 0.404864 Aug 24 10:03:15.267644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 24/49] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:14 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1275 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:03:15.336595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:15.337231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:15.349252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:15.358284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:15.358857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:15.359314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:15.360171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:15.364836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:15.367456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:15.368660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:15.369310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:15.382801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0629be90-15d8-4a42-b7d7-231db0df3028 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1334 microversion: 2.20 time: 0.291094 Aug 24 10:03:15.400072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 26/50] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1334 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:15.868746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:15.889380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-39f328e2-f3ec-4968-90b3-e779d5badfce tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1218 microversion: 2.1 time: 0.557595 Aug 24 10:03:15.908605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 25/51] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:15 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1218 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:15.965942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7c6410a8-a8ee-4f0d-8a3c-2e147a4991b6 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33ddcc430>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:16.093571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-7c6410a8-a8ee-4f0d-8a3c-2e147a4991b6 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] HTTP exception thrown: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7c6410a8-a8ee-4f0d-8a3c-2e147a4991b6 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Request method failure captured: Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 HTTP/1.1 Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdPDX7uTY0d2sh37QEWMwf0NdqDYMaHx9EfWmaeFZm5wIcF-xbNOXXJx656u_-6yzoMUW1hulU9pQP3UT1fF0ktCjyLK3AIaAkRDqv8TQKaQ_6UDFKxIJzDxwnJ7Fe3ztgkOYLJRA8wJXo2n-i4zWuH3HF4sYmSlnm_XW_BD19fND0 Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 9e7aa26bd9e64b30a6c0c3d3824a1dda Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:03:16.096952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 9e7aa26bd9e64b30a6c0c3d3824a1dda Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesSnapshotsTestJSON-1940705297-project Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 2e5d00edb03c4b6c91290f1d4bf4c28c Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1940705297-project Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '67c1c0ca-8964-4e1a-b566-699c6e7456c3'} Aug 24 10:03:16.098245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. (HTTP 404) (Request-ID: req-e5443f62-eb0a-47ea-b007-b0caa4e09a7a) Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:16.099123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:16.100280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 24 10:03:16.101441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7c6410a8-a8ee-4f0d-8a3c-2e147a4991b6 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Returning 404 to user: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:16.102717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7c6410a8-a8ee-4f0d-8a3c-2e147a4991b6 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3" status: 404 len: 109 microversion: 2.1 time: 0.145390 Aug 24 10:03:16.103803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 27/52] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:15 2022] GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 => generated 109 bytes in 147 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:03:16.123065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1c8f7185-b30b-4f0c-b130-38c1baeff2a8 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33e5ec790>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:16.293696 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:16.294487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:16.316116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:16.316723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:16.317026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:16.317661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:16.318274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:16.319500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:16.380864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-1c8f7185-b30b-4f0c-b130-38c1baeff2a8 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] HTTP exception thrown: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1c8f7185-b30b-4f0c-b130-38c1baeff2a8 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Request method failure captured: Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 HTTP/1.1 Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdPDX7uTY0d2sh37QEWMwf0NdqDYMaHx9EfWmaeFZm5wIcF-xbNOXXJx656u_-6yzoMUW1hulU9pQP3UT1fF0ktCjyLK3AIaAkRDqv8TQKaQ_6UDFKxIJzDxwnJ7Fe3ztgkOYLJRA8wJXo2n-i4zWuH3HF4sYmSlnm_XW_BD19fND0 Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9e7aa26bd9e64b30a6c0c3d3824a1dda Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:03:16.383060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9e7aa26bd9e64b30a6c0c3d3824a1dda Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesSnapshotsTestJSON-1940705297-project Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 2e5d00edb03c4b6c91290f1d4bf4c28c Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1940705297-project Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '67c1c0ca-8964-4e1a-b566-699c6e7456c3'} Aug 24 10:03:16.384222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. (HTTP 404) (Request-ID: req-1f1aa799-c6bd-4ef3-950b-4a3f86e5f3bf) Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:16.384976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 24 10:03:16.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 24 10:03:16.390973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1c8f7185-b30b-4f0c-b130-38c1baeff2a8 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Returning 404 to user: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1c8f7185-b30b-4f0c-b130-38c1baeff2a8 tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "DELETE /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3" status: 404 len: 109 microversion: 2.1 time: 0.267336 Aug 24 10:03:16.393442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 26/53] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:03:16 2022] DELETE /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 => generated 109 bytes in 269 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:03:16.394828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-82878b9f-34ea-43c9-a133-e38991145d6a tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Calling method '.version_select at 0x7fa33e5e1ee0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:16.517188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-82878b9f-34ea-43c9-a133-e38991145d6a tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] HTTP exception thrown: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-82878b9f-34ea-43c9-a133-e38991145d6a tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Request method failure captured: Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 HTTP/1.1 Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdPDX7uTY0d2sh37QEWMwf0NdqDYMaHx9EfWmaeFZm5wIcF-xbNOXXJx656u_-6yzoMUW1hulU9pQP3UT1fF0ktCjyLK3AIaAkRDqv8TQKaQ_6UDFKxIJzDxwnJ7Fe3ztgkOYLJRA8wJXo2n-i4zWuH3HF4sYmSlnm_XW_BD19fND0 Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9e7aa26bd9e64b30a6c0c3d3824a1dda Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:03:16.519162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9e7aa26bd9e64b30a6c0c3d3824a1dda"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9e7aa26bd9e64b30a6c0c3d3824a1dda Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1940705297 Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesSnapshotsTestJSON-1940705297-project Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 2e5d00edb03c4b6c91290f1d4bf4c28c Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1940705297-project Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '67c1c0ca-8964-4e1a-b566-699c6e7456c3'} Aug 24 10:03:16.520869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. (HTTP 404) (Request-ID: req-ee5496ee-e043-4f0f-b81e-746bb7074ff1) Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:16.521649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:16.522643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 24 10:03:16.523602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-82878b9f-34ea-43c9-a133-e38991145d6a tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] Returning 404 to user: Volume 67c1c0ca-8964-4e1a-b566-699c6e7456c3 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-82878b9f-34ea-43c9-a133-e38991145d6a tempest-VolumesSnapshotsTestJSON-1940705297 tempest-VolumesSnapshotsTestJSON-1940705297-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3" status: 404 len: 109 microversion: 2.1 time: 0.131404 Aug 24 10:03:16.527108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 27/54] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:16 2022] GET /compute/v2.1/os-volumes/67c1c0ca-8964-4e1a-b566-699c6e7456c3 => generated 109 bytes in 137 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:03:16.542321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:16.545491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:16.562588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:16.570161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:16.570758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:16.575292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:16.578596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:16.579130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:16.850990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:16.866193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:16.866193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:16.866193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a9b57a70-e04f-40e7-94ea-81178a1abba4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1340 microversion: 2.20 time: 0.335568 Aug 24 10:03:16.874310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 28/55] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:16 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1340 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:16.931362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:16.935373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:16.951696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:16.967034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:16.967662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:16.968477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:16.968959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:16.969421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:17.106104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:17.114513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9a1464cd-63d3-4d6b-a741-ce3d380324a5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1273 microversion: 2.1 time: 0.825109 Aug 24 10:03:17.116734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 28/56] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:16 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1273 bytes in 828 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:03:17.252767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:17.263157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d192c9d7-5c65-4ec2-af3b-a529e9c9d3f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1280 microversion: 2.1 time: 0.335558 Aug 24 10:03:17.284675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 29/57] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:16 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1280 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:17.900297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:17.901142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:17.929931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:17.932184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:17.933208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:17.943041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:17.946338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:17.948625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:18.092935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:18.109860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:18.120786 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:18.123387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9b74bc70-8970-4059-8c00-66208e644aee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1340 microversion: 2.20 time: 0.227276 Aug 24 10:03:18.124981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 29/58] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1340 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:18.154318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:18.155605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:18.177343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:18.178649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:18.179491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:18.180620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:18.181358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:18.182441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:18.292753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:18.293753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:18.305956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:18.307011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:18.309482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:18.310400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:18.312384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:18.313003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:18.436282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:18.447013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-05212f53-d90d-4379-acd4-05e7086c822f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1280 microversion: 2.1 time: 0.159937 Aug 24 10:03:18.448906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 30/59] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:18 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1280 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:18.677738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:18.692395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fbb0a036-e6bc-43ae-ad4d-4970eaa6b07e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.557113 Aug 24 10:03:18.693359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 30/60] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:18 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:19.149774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:19.150587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:19.164300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:19.166801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:19.168552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:19.169318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:19.169796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:19.170230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:19.817365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-734a9685-4bfe-4e0d-a759-242812d785d4 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Action: 'create', calling method: .version_select at 0x7fa33e5ec1f0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-275606318", "metadata": {"Type": "work"}}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-734a9685-4bfe-4e0d-a759-242812d785d4 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Request method failure captured: Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 121 Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdnRuvStVhylB8euxc-CtncpjsSGtoZZebLPuy21IgYcVczOE1BZX5dXPVXSAeL7zzgBy_SxX_PJ-4PS0St-yHrl-Tq6BDCE0t0PUg-U0JAZh_YxwT85bkKk-BRlxgBhBUc4u3_qKWSIrHLWV1JZZaljOLCkENNlJqjnEf_R19hXVU Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:03:19.824405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_cb3ad0552e544f35b45820c96d033974"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: a5ce68d8ee3347cd93b8c240e8a3245c Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-275606318", "metadata": {"Type": "work"}}} Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: Aug 24 10:03:19.825861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-275606318', 'metadata': {'Type': 'work'}}}} Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise error Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi '#$%' Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:19.826879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 24 10:03:19.828408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 24 10:03:19.829922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 24 10:03:19.829922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:19.832698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-734a9685-4bfe-4e0d-a759-242812d785d4 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:19.833879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-734a9685-4bfe-4e0d-a759-242812d785d4 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] 10.222.0.86 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.584901 Aug 24 10:03:19.834984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 31/61] 10.222.0.86 () {60 vars in 1156 bytes} [Wed Aug 24 10:03:19 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 586 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Aug 24 10:03:19.843742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:19.845223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:19.856020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:19.878779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:19.879206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:19.880312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:19.882676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:19.883861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:19.884796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:19.885319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:19.887631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:19.889574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-172b47e3-c29b-45ee-937e-4e76e3541695 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.742891 Aug 24 10:03:19.892784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 31/62] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 749 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:19.899263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:19.900284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:19.916555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:19.922312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:19.923636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:19.924692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:19.925771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:19.926810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:20.059604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:20.085360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-33a6f017-f14c-49df-ba1b-5d0ae1125a31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1280 microversion: 2.1 time: 0.248974 Aug 24 10:03:20.092140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 32/63] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:19 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1280 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:20.107343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6a1b9bb4-3718-44dd-b5a5-5d89c7959709 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Action: 'create', calling method: .version_select at 0x7fa33e5ec1f0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-398453049", "metadata": {"Type": "work"}}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6a1b9bb4-3718-44dd-b5a5-5d89c7959709 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Request method failure captured: Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 119 Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdnRuvStVhylB8euxc-CtncpjsSGtoZZebLPuy21IgYcVczOE1BZX5dXPVXSAeL7zzgBy_SxX_PJ-4PS0St-yHrl-Tq6BDCE0t0PUg-U0JAZh_YxwT85bkKk-BRlxgBhBUc4u3_qKWSIrHLWV1JZZaljOLCkENNlJqjnEf_R19hXVU Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:03:20.133334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_cb3ad0552e544f35b45820c96d033974"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: a5ce68d8ee3347cd93b8c240e8a3245c Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-398453049", "metadata": {"Type": "work"}}} Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: Aug 24 10:03:20.134877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-398453049', 'metadata': {'Type': 'work'}}}} Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise error Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi '0' Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:20.135708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 24 10:03:20.137059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 24 10:03:20.138328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 24 10:03:20.138328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.139833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6a1b9bb4-3718-44dd-b5a5-5d89c7959709 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:20.141709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6a1b9bb4-3718-44dd-b5a5-5d89c7959709 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] 10.222.0.86 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.048186 Aug 24 10:03:20.142315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 33/64] 10.222.0.86 () {60 vars in 1156 bytes} [Wed Aug 24 10:03:20 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 50 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:03:20.159313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8075e946-919a-4c55-94f3-0fa22796238e tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Action: 'create', calling method: .version_select at 0x7fa33e561f70>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-337953982", "metadata": {"Type": "work"}}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8075e946-919a-4c55-94f3-0fa22796238e tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Request method failure captured: Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 118 Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdnRuvStVhylB8euxc-CtncpjsSGtoZZebLPuy21IgYcVczOE1BZX5dXPVXSAeL7zzgBy_SxX_PJ-4PS0St-yHrl-Tq6BDCE0t0PUg-U0JAZh_YxwT85bkKk-BRlxgBhBUc4u3_qKWSIrHLWV1JZZaljOLCkENNlJqjnEf_R19hXVU Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:03:20.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_cb3ad0552e544f35b45820c96d033974"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: a5ce68d8ee3347cd93b8c240e8a3245c Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-337953982", "metadata": {"Type": "work"}}} Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: Aug 24 10:03:20.176216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-337953982', 'metadata': {'Type': 'work'}}}} Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise error Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi '' Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:20.177044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 24 10:03:20.179972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 24 10:03:20.183715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 24 10:03:20.183715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.185067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8075e946-919a-4c55-94f3-0fa22796238e tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:20.186337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8075e946-919a-4c55-94f3-0fa22796238e tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] 10.222.0.86 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.037233 Aug 24 10:03:20.187591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 34/65] 10.222.0.86 () {60 vars in 1156 bytes} [Wed Aug 24 10:03:20 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 39 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:03:20.201175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-94529b69-6564-420e-afdc-229068054521 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Calling method '.version_select at 0x7fa33e5ffc10>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:20.301195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-94529b69-6564-420e-afdc-229068054521 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] HTTP exception thrown: Volume tempest-invalid-1203276677 could not be found. Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-94529b69-6564-420e-afdc-229068054521 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Request method failure captured: Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1203276677 HTTP/1.1 Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdnRuvStVhylB8euxc-CtncpjsSGtoZZebLPuy21IgYcVczOE1BZX5dXPVXSAeL7zzgBy_SxX_PJ-4PS0St-yHrl-Tq6BDCE0t0PUg-U0JAZh_YxwT85bkKk-BRlxgBhBUc4u3_qKWSIrHLWV1JZZaljOLCkENNlJqjnEf_R19hXVU Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:03:20.305776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_cb3ad0552e544f35b45820c96d033974"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: a5ce68d8ee3347cd93b8c240e8a3245c Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Volume tempest-invalid-1203276677 could not be found. Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': 'tempest-invalid-1203276677'} Aug 24 10:03:20.307272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1203276677 could not be found. (HTTP 404) (Request-ID: req-43db0971-52b6-4d79-bedb-d9519d845a36) Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.308126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 24 10:03:20.309253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1203276677 could not be found. Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:03:20.310365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1203276677 could not be found. Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1203276677 could not be found. Aug 24 10:03:20.311564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.315952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-94529b69-6564-420e-afdc-229068054521 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Returning 404 to user: Volume tempest-invalid-1203276677 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:20.317209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-94529b69-6564-420e-afdc-229068054521 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] 10.222.0.86 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1203276677" status: 404 len: 99 microversion: 2.1 time: 0.122821 Aug 24 10:03:20.322203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 35/66] 10.222.0.86 () {58 vars in 1218 bytes} [Wed Aug 24 10:03:20 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1203276677 => generated 99 bytes in 129 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Aug 24 10:03:20.342922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-02c8ff4b-7638-4469-8ed5-df734e24538c tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] 10.222.0.86 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.016253 Aug 24 10:03:20.346743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 36/67] 10.222.0.86 () {58 vars in 1140 bytes} [Wed Aug 24 10:03:20 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 22 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Aug 24 10:03:20.374223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1a884e16-b913-4e4e-b737-9c42a200ccd5 tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.017640 Aug 24 10:03:20.376709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 37/68] 10.222.0.86 () {58 vars in 1137 bytes} [Wed Aug 24 10:03:20 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 23 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Aug 24 10:03:20.403125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4bcdb1aa-96bf-4e65-926e-b54c82a009ca tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Calling method '.version_select at 0x7fa33e5eca60>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:20.520038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-4bcdb1aa-96bf-4e65-926e-b54c82a009ca tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] HTTP exception thrown: Volume 61bf0c32-992a-4afe-b9fc-967c05b30765 could not be found. Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4bcdb1aa-96bf-4e65-926e-b54c82a009ca tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Request method failure captured: Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/os-volumes/61bf0c32-992a-4afe-b9fc-967c05b30765 HTTP/1.1 Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdnRuvStVhylB8euxc-CtncpjsSGtoZZebLPuy21IgYcVczOE1BZX5dXPVXSAeL7zzgBy_SxX_PJ-4PS0St-yHrl-Tq6BDCE0t0PUg-U0JAZh_YxwT85bkKk-BRlxgBhBUc4u3_qKWSIrHLWV1JZZaljOLCkENNlJqjnEf_R19hXVU Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:03:20.522441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_cb3ad0552e544f35b45820c96d033974"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: a5ce68d8ee3347cd93b8c240e8a3245c Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Volume 61bf0c32-992a-4afe-b9fc-967c05b30765 could not be found. Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '61bf0c32-992a-4afe-b9fc-967c05b30765'} Aug 24 10:03:20.523541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 61bf0c32-992a-4afe-b9fc-967c05b30765 could not be found. (HTTP 404) (Request-ID: req-b4022fa7-af38-4543-80a0-4ce8174223e0) Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.524274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 24 10:03:20.525253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 61bf0c32-992a-4afe-b9fc-967c05b30765 could not be found. Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 24 10:03:20.526227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 61bf0c32-992a-4afe-b9fc-967c05b30765 could not be found. Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 61bf0c32-992a-4afe-b9fc-967c05b30765 could not be found. Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4bcdb1aa-96bf-4e65-926e-b54c82a009ca tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Returning 404 to user: Volume 61bf0c32-992a-4afe-b9fc-967c05b30765 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4bcdb1aa-96bf-4e65-926e-b54c82a009ca tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] 10.222.0.86 "DELETE /compute/v2.1/os-volumes/61bf0c32-992a-4afe-b9fc-967c05b30765" status: 404 len: 109 microversion: 2.1 time: 0.142053 Aug 24 10:03:20.527184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 38/69] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:03:20 2022] DELETE /compute/v2.1/os-volumes/61bf0c32-992a-4afe-b9fc-967c05b30765 => generated 109 bytes in 144 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:03:20.549961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7cdfd9cc-744a-4e91-a2fa-d69b00e16e9d tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Calling method '.version_select at 0x7fa33e590dc0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:20.553784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:20.571645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6184d6fc-c91c-466e-b3da-61e31942ca51 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.675053 Aug 24 10:03:20.588846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 32/70] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:19 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:20.619020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-7cdfd9cc-744a-4e91-a2fa-d69b00e16e9d tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] HTTP exception thrown: Volume 63b19848-42d2-4019-94af-30558142d298 could not be found. Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7cdfd9cc-744a-4e91-a2fa-d69b00e16e9d tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Request method failure captured: Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/os-volumes/63b19848-42d2-4019-94af-30558142d298 HTTP/1.1 Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdnRuvStVhylB8euxc-CtncpjsSGtoZZebLPuy21IgYcVczOE1BZX5dXPVXSAeL7zzgBy_SxX_PJ-4PS0St-yHrl-Tq6BDCE0t0PUg-U0JAZh_YxwT85bkKk-BRlxgBhBUc4u3_qKWSIrHLWV1JZZaljOLCkENNlJqjnEf_R19hXVU Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:03:20.623728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_cb3ad0552e544f35b45820c96d033974"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/cb3ad0552e544f35b45820c96d033974"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: cb3ad0552e544f35b45820c96d033974 Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesNegativeTest-2093095556 Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: a5ce68d8ee3347cd93b8c240e8a3245c Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesNegativeTest-2093095556-project Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Volume 63b19848-42d2-4019-94af-30558142d298 could not be found. Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '63b19848-42d2-4019-94af-30558142d298'} Aug 24 10:03:20.625889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 63b19848-42d2-4019-94af-30558142d298 could not be found. (HTTP 404) (Request-ID: req-489787b2-459d-484a-b0be-4d3bc5e5886c) Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.627153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 24 10:03:20.628706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 63b19848-42d2-4019-94af-30558142d298 could not be found. Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 24 10:03:20.629922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 63b19848-42d2-4019-94af-30558142d298 could not be found. Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 63b19848-42d2-4019-94af-30558142d298 could not be found. Aug 24 10:03:20.631298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:03:20.634454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7cdfd9cc-744a-4e91-a2fa-d69b00e16e9d tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] Returning 404 to user: Volume 63b19848-42d2-4019-94af-30558142d298 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:03:20.636568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7cdfd9cc-744a-4e91-a2fa-d69b00e16e9d tempest-VolumesNegativeTest-2093095556 tempest-VolumesNegativeTest-2093095556-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/63b19848-42d2-4019-94af-30558142d298" status: 404 len: 109 microversion: 2.1 time: 0.096624 Aug 24 10:03:20.637356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 39/71] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:03:20 2022] GET /compute/v2.1/os-volumes/63b19848-42d2-4019-94af-30558142d298 => generated 109 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:03:20.917835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:20.918814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:20.935998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:20.937087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:20.937753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:20.939352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:20.940214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:20.940911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:21.114249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:21.118729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:21.160683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:21.161251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:21.161705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:21.162538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:21.175380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:21.176796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:21.687930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:21.717178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9a2d0583-03ac-49d7-b2a8-686fb68a5410 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1280 microversion: 2.1 time: 0.607309 Aug 24 10:03:21.720787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 40/72] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:21 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1280 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:21.735466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:21.737259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:21.755823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:21.778727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:21.785872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:21.788946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:21.789831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:21.790203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:21.793451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dc940b6d-4e51-47a3-9f34-0607f0081990 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.879805 Aug 24 10:03:21.795952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:21.797924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:21.800814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:21.801109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 33/73] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:20 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 888 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:22.752199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:22.753002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:22.761003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:22.771321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:22.775286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:22.777242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:22.779766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:22.780404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:22.783705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:22.833131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:22.841873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-150c9399-e796-4c00-8ee8-16118343066e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 1.119571 Aug 24 10:03:22.842402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 41/74] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:21 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 1121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:22.874700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:22.876134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:22.891712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:22.892632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:22.893457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:22.894478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:22.895540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:22.896283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:23.241788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:23.259529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-118eed29-e818-4b6d-b68c-56b841ac124c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.516135 Aug 24 10:03:23.262390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 34/75] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:22 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:23.585769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:23.613184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:23.615363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:23.617100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ef004c0a-342d-46f7-acc2-b6265dbda418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.773458 Aug 24 10:03:23.635038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 42/76] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:22 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 792 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:23.867798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:23.868442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:23.878724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:23.879209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:23.879704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:23.883792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:23.884319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:23.884760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:24.042277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-924d351d-3487-4088-a341-cd5e5e81afa4 req-8a901744-ae60-4102-b401-3f2189329df0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ab73a23-282f-4f54-9a62-5a417d7e7603", "tag": "e2175a9a-3243-44ea-896c-f63d59862a02"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:24.068923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-924d351d-3487-4088-a341-cd5e5e81afa4 req-8a901744-ae60-4102-b401-3f2189329df0 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:24.069398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-924d351d-3487-4088-a341-cd5e5e81afa4 req-8a901744-ae60-4102-b401-3f2189329df0 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:24.070963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-924d351d-3487-4088-a341-cd5e5e81afa4 req-8a901744-ae60-4102-b401-3f2189329df0 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:24.124662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:24.126832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=16 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:24.135471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-924d351d-3487-4088-a341-cd5e5e81afa4 req-8a901744-ae60-4102-b401-3f2189329df0 service nova] Creating event network-changed:e2175a9a-3243-44ea-896c-f63d59862a02 for instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 on hv-cinder-85427 Aug 24 10:03:24.136127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-924d351d-3487-4088-a341-cd5e5e81afa4 req-8a901744-ae60-4102-b401-3f2189329df0 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120653) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 24 10:03:24.136525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-924d351d-3487-4088-a341-cd5e5e81afa4 req-8a901744-ae60-4102-b401-3f2189329df0 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120653) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 24 10:03:24.139654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1e558c7-674c-4848-bf4d-ca6da8004cd7 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:24.140267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1e558c7-674c-4848-bf4d-ca6da8004cd7 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:24.141642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1e558c7-674c-4848-bf4d-ca6da8004cd7 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:24.143948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1e558c7-674c-4848-bf4d-ca6da8004cd7 None None] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:24.145634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1e558c7-674c-4848-bf4d-ca6da8004cd7 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:24.148278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1e558c7-674c-4848-bf4d-ca6da8004cd7 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:24.180854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.compute.rpcapi [None req-c1e558c7-674c-4848-bf4d-ca6da8004cd7 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Aug 24 10:03:24.183391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1e558c7-674c-4848-bf4d-ca6da8004cd7 None None] Releasing lock "compute-rpcapi-router" {{(pid=120653) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 24 10:03:24.199156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:24.211791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-35223082-5d39-46a3-bf06-f4e1f04393a1 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.346004 Aug 24 10:03:24.212432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 35/77] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:23 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:24.234113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c1e558c7-674c-4848-bf4d-ca6da8004cd7 None None] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.313433 Aug 24 10:03:24.241019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 43/78] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:03:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:24.281025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:24.281871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:24.294205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:24.295007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:24.295745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:24.297285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:24.297890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:24.298583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:24.317399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=20 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:24.504904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:24.513487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8d434e06-3caa-485f-9256-e2f1cd6d1c73 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.236325 Aug 24 10:03:24.514059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 36/79] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:24 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:24.529220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-b6b91fcf-22c1-4b7d-9e29-e44c7c683032 req-45492732-0269-401b-afa1-c22f50ab87e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1", "tag": "1b9069bd-2630-4b65-9e91-d7695e3b6a8c"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:24.538558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b6b91fcf-22c1-4b7d-9e29-e44c7c683032 req-45492732-0269-401b-afa1-c22f50ab87e6 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:24.539032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b6b91fcf-22c1-4b7d-9e29-e44c7c683032 req-45492732-0269-401b-afa1-c22f50ab87e6 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:24.539453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b6b91fcf-22c1-4b7d-9e29-e44c7c683032 req-45492732-0269-401b-afa1-c22f50ab87e6 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:24.550987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-b6b91fcf-22c1-4b7d-9e29-e44c7c683032 req-45492732-0269-401b-afa1-c22f50ab87e6 service nova] Creating event network-changed:1b9069bd-2630-4b65-9e91-d7695e3b6a8c for instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 on hv-cinder-85427 Aug 24 10:03:24.555650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-b6b91fcf-22c1-4b7d-9e29-e44c7c683032 req-45492732-0269-401b-afa1-c22f50ab87e6 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.265420 Aug 24 10:03:24.559541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 44/80] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:03:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:24.638743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:24.639357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:24.645508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:24.646043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:24.646531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:24.647235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:24.647654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:24.648075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:24.815155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:24.844587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:24.846099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:24.847195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-456fb6df-e16f-437f-89f0-90cd8b3ddc4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.212419 Aug 24 10:03:24.847885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 37/81] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:24 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:25.236907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:25.243096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:25.253430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:25.254074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:25.254759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:25.255606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:25.256206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:25.256753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:25.533505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:25.539252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:25.539864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:25.550585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:25.551076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:25.551509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:25.552241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:25.552690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:25.553114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-485ddc8c-1131-494f-b1cc-c3baa9fda991 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.313956 Aug 24 10:03:25.553240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:25.556980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 45/82] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:25 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:25.873291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:25.874681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:25.886083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:25.889813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:25.891414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:25.893942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:25.895914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:25.897252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:25.924614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:25.934234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9e22de25-fb73-4579-99d0-6df78171df44 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.400415 Aug 24 10:03:25.936984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 38/83] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:25 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:26.575451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:26.576275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:26.596059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:26.598052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:26.599901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:26.602759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:26.604167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:26.604612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:26.880003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:26.890174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:26.897037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:26.900307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8481e06c-ef62-4f1a-8f80-b41e3438afad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 1.031808 Aug 24 10:03:26.904122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 46/84] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:25 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 1036 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:26.915573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-a7533119-51be-4786-883d-b00e7d5cbddd req-92e95e10-200a-4498-aa4a-fcec9e77ab02 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a4d2604d-01d1-4c1f-b02b-0c62ae2a027d", "tag": "1a686d3f-cefc-4297-b3cd-7ebf8ce50b96"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:26.939521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a7533119-51be-4786-883d-b00e7d5cbddd req-92e95e10-200a-4498-aa4a-fcec9e77ab02 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:26.942415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a7533119-51be-4786-883d-b00e7d5cbddd req-92e95e10-200a-4498-aa4a-fcec9e77ab02 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:26.944024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a7533119-51be-4786-883d-b00e7d5cbddd req-92e95e10-200a-4498-aa4a-fcec9e77ab02 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:26.986302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-a7533119-51be-4786-883d-b00e7d5cbddd req-92e95e10-200a-4498-aa4a-fcec9e77ab02 service nova] Creating event network-changed:1a686d3f-cefc-4297-b3cd-7ebf8ce50b96 for instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d on hv-cinder-85427 Aug 24 10:03:27.005385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-a7533119-51be-4786-883d-b00e7d5cbddd req-92e95e10-200a-4498-aa4a-fcec9e77ab02 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099326 Aug 24 10:03:27.012909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 47/85] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:03:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:27.034044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:27.034366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:27.059779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:27.060333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:27.060789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:27.062060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:27.062060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:27.069390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:27.861301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:27.873100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-967a2bb1-611e-4f54-ba18-ec29e9653f52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.855466 Aug 24 10:03:27.876821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 48/86] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:27 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 860 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:27.932985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:27.934853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:27.947022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:27.948680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:27.950631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:27.952794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:27.955977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:27.957073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:28.047844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:28.063731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6299137a-3b4b-4a9a-b8db-1c2d604b1abb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 1.490846 Aug 24 10:03:28.066066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 39/87] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:26 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 1495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:03:28.326113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:28.335527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:28.337324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:28.338458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b733fa18-bedf-438f-b0c4-5c3244df89f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.411529 Aug 24 10:03:28.339391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 49/88] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:27 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:03:28.628631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-0b787fed-2b9a-4f41-b692-830ee9ae3c97 req-40fa5e64-9c53-45ff-a074-310b05998dbc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d6a8cae5-8006-46a9-90b3-4556894041b0", "tag": "5930ef7d-e5d9-40c3-bc98-18c0ec1f1de4"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:28.650586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-0b787fed-2b9a-4f41-b692-830ee9ae3c97 req-40fa5e64-9c53-45ff-a074-310b05998dbc service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:28.651337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-0b787fed-2b9a-4f41-b692-830ee9ae3c97 req-40fa5e64-9c53-45ff-a074-310b05998dbc service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:28.652238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-0b787fed-2b9a-4f41-b692-830ee9ae3c97 req-40fa5e64-9c53-45ff-a074-310b05998dbc service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:28.674991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-0b787fed-2b9a-4f41-b692-830ee9ae3c97 req-40fa5e64-9c53-45ff-a074-310b05998dbc service nova] Creating event network-changed:5930ef7d-e5d9-40c3-bc98-18c0ec1f1de4 for instance d6a8cae5-8006-46a9-90b3-4556894041b0 on hv-cinder-85427 Aug 24 10:03:28.675733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-0b787fed-2b9a-4f41-b692-830ee9ae3c97 req-40fa5e64-9c53-45ff-a074-310b05998dbc service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120654) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 24 10:03:28.676242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-0b787fed-2b9a-4f41-b692-830ee9ae3c97 req-40fa5e64-9c53-45ff-a074-310b05998dbc service nova] Acquired lock "compute-rpcapi-router" {{(pid=120654) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 24 10:03:28.677467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-727b9db1-13e4-44e3-b674-e842ba5e3c62 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:28.678064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-727b9db1-13e4-44e3-b674-e842ba5e3c62 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:28.678637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-727b9db1-13e4-44e3-b674-e842ba5e3c62 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:28.679375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-727b9db1-13e4-44e3-b674-e842ba5e3c62 None None] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:28.679965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-727b9db1-13e4-44e3-b674-e842ba5e3c62 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:28.680460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-727b9db1-13e4-44e3-b674-e842ba5e3c62 None None] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:28.690789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.compute.rpcapi [None req-727b9db1-13e4-44e3-b674-e842ba5e3c62 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Aug 24 10:03:28.691758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-727b9db1-13e4-44e3-b674-e842ba5e3c62 None None] Releasing lock "compute-rpcapi-router" {{(pid=120654) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 24 10:03:28.713612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-727b9db1-13e4-44e3-b674-e842ba5e3c62 None None] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091853 Aug 24 10:03:28.715165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 40/89] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:03:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:28.900644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:28.901256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:28.908234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:28.908717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:28.909345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:28.910044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:28.910495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:28.910914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:29.081628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:29.090355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:29.091054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:29.091407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f269fe7d-e8e2-4deb-8f62-b7b3e3deee9d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.193538 Aug 24 10:03:29.098162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:29.098635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:29.099074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:29.099783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:29.100532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:29.100956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:29.105088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 50/90] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:28 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:29.105326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:29.296314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:29.311489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2c620b82-fce2-43cd-a305-fa0437416475 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.225223 Aug 24 10:03:29.312936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 41/91] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:29 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:29.364655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:29.365938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:29.372345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:29.372957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:29.373532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:29.374259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:29.374708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:29.376479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:29.395830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:29.521442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:29.529632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:29.531763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:29.532888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-91195f4c-e49e-4e02-9379-a5649896fd80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.173181 Aug 24 10:03:29.533430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 51/92] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:30.114810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:30.115422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:30.121466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:30.121953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:30.122577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:30.123276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:30.123729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:30.124628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:30.319449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:30.334308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:30.335491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:30.339310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6a1ac85b-190d-4fbd-8bbc-ef84777917f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.228523 Aug 24 10:03:30.340484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 42/93] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:30 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:30.355834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:30.356364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:30.356799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:30.357498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:30.357969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:30.358579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:30.507921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:30.516865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b566e8d4-a362-4cc0-8e2d-918b74edf452 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.186852 Aug 24 10:03:30.517427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 52/94] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:30 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:30.556502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:30.557167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:30.564367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:30.564844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:30.565465 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:30.566338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:30.566941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:30.567362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:30.713728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:30.723010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:30.724803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:30.726032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b723d7c1-fc17-4508-8e74-b6233884fbb8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.173988 Aug 24 10:03:30.726670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 43/95] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:31.367880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:31.371846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:31.394756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:31.395941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:31.396810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:31.398182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:31.398953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:31.415810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:31.539361 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:31.540407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:31.549600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:31.550948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:31.551614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:31.561465 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:31.563139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:31.567920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:31.708079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:31.733823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5b92d04c-b660-4f06-9d30-d29cea2c2d10 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.375739 Aug 24 10:03:31.734457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 53/96] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:31 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:31.768297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:31.770282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:31.791268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:31.792160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:31.793399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:31.795513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:31.798934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:31.802807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:31.986327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:31.997699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7ee483e7-cb01-4628-ae7c-47db2ad5a5a6 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.463812 Aug 24 10:03:32.003964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 44/97] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:31 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:32.692732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:32.705898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:32.708628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:32.711581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6e8d1dde-2fcc-46ce-84a3-4e0b8b762444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.964985 Aug 24 10:03:32.724179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 54/98] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:31 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 975 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:32.767861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:32.768656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:32.776327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:32.783745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:32.788295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:32.789172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:32.789735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:32.795719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:33.021011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:33.021720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:33.033098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:33.033384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:33.033886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:33.034653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:33.036034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:33.041833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:33.332740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:33.349304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-65c80deb-7d58-4dab-bf25-13cee7b0a738 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.586013 Aug 24 10:03:33.353814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 45/99] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:32 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:33.751723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:33.752628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:33.774072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:33.781863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:33.786149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:33.787054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:33.793872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:33.796960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:33.893843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:33.903046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-44fc350d-f44c-4b09-b2f9-e44c0f64521d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.885742 Aug 24 10:03:33.905559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 55/100] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:33 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 889 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:03:34.210857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:34.225099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:34.228232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:34.229983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-244629ec-9ab1-4957-a765-96ac45923ad3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.483011 Aug 24 10:03:34.230535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 46/101] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:03:34.379109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:34.380297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:34.382739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:34.401138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:34.401642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:34.405557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:34.406580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:34.406974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:34.407417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:34.681714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:34.709947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6c8dc898-8a30-4095-9aa5-f14309a98b8c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.336941 Aug 24 10:03:34.711648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 56/102] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:34 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:34.929877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:34.933853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:34.943299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:34.944200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:34.945112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:34.946993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:34.947638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:34.948276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:35.133690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:35.144494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6e7fdcad-d288-4b2f-9ca1-3540da51c7e5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.220289 Aug 24 10:03:35.159997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 47/103] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:34 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:35.253575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:35.254936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:35.261690 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:35.279384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:35.279384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:35.279384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:35.279605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:35.280031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:35.516432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:35.533326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:35.537580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:35.540945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-815faf79-08b2-4884-96ee-3da04d876d73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.289100 Aug 24 10:03:35.556437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 57/104] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:35 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:35.732657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:35.733271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:35.736563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:35.741718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:35.742751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:35.743401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:35.744736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:35.745241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:35.745726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:35.919436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:35.927831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5fee5ace-ee09-4e20-9602-cd1b00ad36fd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.199651 Aug 24 10:03:35.928436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 48/105] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:35 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:36.170213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:36.170891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:36.181359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:36.184897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:36.185385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:36.186135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:36.186627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:36.187614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:36.433644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:36.445236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f9481415-20b1-4ef8-90ab-a38fc45645f9 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.282042 Aug 24 10:03:36.447959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 58/106] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:36 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:36.567931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:36.568580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:36.575869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:36.577031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:36.578521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:36.580125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:36.580933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:36.581449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:36.827761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:36.845366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:36.846923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:36.849526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5980c069-2529-4f23-82ee-0cae4f5446b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.285161 Aug 24 10:03:36.851196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 49/107] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:36 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:36.952428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:36.955193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:36.962607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=6 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:36.964987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:36.965491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:36.966656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:36.967375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:36.967831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:36.968881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:37.241722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:37.252972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8d663a09-8fb8-4c24-aac9-cadf95f0b153 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.307686 Aug 24 10:03:37.253704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 59/108] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:36 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:37.481152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:37.483100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:37.495170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:37.495887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:37.496662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:37.497793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:37.498528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:37.499216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:37.691322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:37.701208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e1fad4f0-ea95-4c73-9e11-77c0f1fe8efe tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.228577 Aug 24 10:03:37.716877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 50/109] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:37 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:37.875414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:37.876117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:37.888678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:37.890971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:37.891427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:37.892139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:37.893596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:37.896353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:38.278755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:38.279448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:38.293791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:38.294444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:38.295030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:38.295886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:38.297946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:38.301225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:38.432823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:38.448413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:38.450265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:38.451475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a275b386-cc22-41c0-b8a2-ce500763ab4d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.580755 Aug 24 10:03:38.458546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 60/110] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:38.642561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:38.656889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-78f5904e-d327-4637-aed1-01982ee41759 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.380501 Aug 24 10:03:38.683685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 51/111] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:38 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:38.726216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:38.727118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:38.739006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:38.739775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:38.741571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:38.745864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:38.747610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:38.749142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:38.930270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:38.939626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6f3860c2-ebe7-4ea7-a3ba-a03bb682f2f2 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.217810 Aug 24 10:03:38.940382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 61/112] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:38 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:39.477203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:39.477816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:39.480660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:39.488465 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:39.488952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:39.489380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:39.490065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:39.497859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:39.498403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:39.670517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-d10829ce-6ff0-4e05-893b-22d85adeff5e req-2a964589-7757-4d92-9e96-4a14d330856b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ab73a23-282f-4f54-9a62-5a417d7e7603", "name": "network-vif-plugged", "status": "completed", "tag": "e2175a9a-3243-44ea-896c-f63d59862a02"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:39.685194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:39.692448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-d10829ce-6ff0-4e05-893b-22d85adeff5e req-2a964589-7757-4d92-9e96-4a14d330856b service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:39.693250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-d10829ce-6ff0-4e05-893b-22d85adeff5e req-2a964589-7757-4d92-9e96-4a14d330856b service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:39.694960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-d10829ce-6ff0-4e05-893b-22d85adeff5e req-2a964589-7757-4d92-9e96-4a14d330856b service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:39.702041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:39.708298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:39.709696 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1f34bd9a-cf45-46c6-9139-9fa4236f855f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.237598 Aug 24 10:03:39.711990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 52/113] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:39 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:39.725334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:39.727969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:39.738773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-d10829ce-6ff0-4e05-893b-22d85adeff5e req-2a964589-7757-4d92-9e96-4a14d330856b service nova] Creating event network-vif-plugged:e2175a9a-3243-44ea-896c-f63d59862a02 for instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 on hv-cinder-85427 Aug 24 10:03:39.747677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:39.748499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:39.751359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:39.752368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:39.753766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:39.754905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-d10829ce-6ff0-4e05-893b-22d85adeff5e req-2a964589-7757-4d92-9e96-4a14d330856b service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.174565 Aug 24 10:03:39.756121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:39.756820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 62/114] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:03:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:03:39.963966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:39.965078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:39.985881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:39.988539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:39.990489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:39.991483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:39.993495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:39.994448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:40.094864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:40.103517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4f417907-9a5d-4358-94b4-155595e766bf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.389493 Aug 24 10:03:40.113153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 53/115] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:39 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:40.321591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:40.341698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-232535ca-ac4f-4e1f-86ef-488238fe1b37 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.382549 Aug 24 10:03:40.346285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 63/116] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:39 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:40.735763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:40.736610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:40.751048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:40.751639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:40.753087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:40.754297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:40.756567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:40.758157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:40.936401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:40.948367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:40.951003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:40.952437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2f3aa7f0-aa2c-44e7-9938-983f38252438 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.222255 Aug 24 10:03:40.967120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 54/117] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:41.134763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:41.135386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:41.149113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:41.149797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:41.150238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:41.151107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:41.151728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:41.152187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:41.301330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:41.310649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8835100f-60c7-4473-ba98-df255449807d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.180193 Aug 24 10:03:41.311560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 64/118] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:41 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:41.367546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:41.368226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:41.374532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:41.375058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:41.375647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:41.376461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:41.377259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:41.377967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:41.528706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:41.537100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-46b2482a-9516-49a7-b81d-72531e2e1f2e tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.173615 Aug 24 10:03:41.537667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 55/119] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:41 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:41.975245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:41.976516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:41.984107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:41.985171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:41.986223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:41.988307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:41.988642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:41.989175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:42.153903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:42.188214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:42.189728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:42.190849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d306830a-0b0e-4990-bb9c-198620b21a90 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.219607 Aug 24 10:03:42.194138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 65/120] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:41 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:42.335160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:42.336586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:42.347467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:42.351774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:42.352510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:42.353349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:42.355017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:42.356506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:42.529847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:42.541625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2db212c9-ab45-4686-b6c9-373ca3019144 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.212369 Aug 24 10:03:42.542206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 56/121] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:42 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:42.559336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:42.559983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:42.571018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:42.572759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:42.573751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:42.576743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:42.578716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:42.581787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:42.657370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-8c761446-6f25-4b59-8691-5dd1f1902a97 req-cf42089d-099c-4919-bacc-0a68c486c89a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a4d2604d-01d1-4c1f-b02b-0c62ae2a027d", "name": "network-vif-plugged", "status": "completed", "tag": "1a686d3f-cefc-4297-b3cd-7ebf8ce50b96"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:42.674934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-8c761446-6f25-4b59-8691-5dd1f1902a97 req-cf42089d-099c-4919-bacc-0a68c486c89a service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:42.676649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-8c761446-6f25-4b59-8691-5dd1f1902a97 req-cf42089d-099c-4919-bacc-0a68c486c89a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:42.677844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-8c761446-6f25-4b59-8691-5dd1f1902a97 req-cf42089d-099c-4919-bacc-0a68c486c89a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:42.715120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-8c761446-6f25-4b59-8691-5dd1f1902a97 req-cf42089d-099c-4919-bacc-0a68c486c89a service nova] Creating event network-vif-plugged:1a686d3f-cefc-4297-b3cd-7ebf8ce50b96 for instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d on hv-cinder-85427 Aug 24 10:03:42.728270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-8c761446-6f25-4b59-8691-5dd1f1902a97 req-cf42089d-099c-4919-bacc-0a68c486c89a service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074096 Aug 24 10:03:42.730483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 57/122] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:03:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:42.845178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:42.855452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f8bed87b-55de-4b8d-9738-81ba38277da8 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.300850 Aug 24 10:03:42.866186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 66/123] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:42 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:43.230568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:43.231940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:43.245906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:43.246989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:43.247654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:43.249953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:43.251319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:43.252024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:43.295100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:43.503381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:43.513351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:43.514558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:43.515772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ab85ca35-1881-4ea3-b5eb-b9f2351396c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.301817 Aug 24 10:03:43.517160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 58/124] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:03:43.566126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:43.567396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:43.574656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:43.575260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:43.575694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:43.576411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:43.576863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:43.577457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:43.714191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:43.724845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4c732414-37f5-42b9-b4f5-b2c5c3b06763 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.163193 Aug 24 10:03:43.726269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 67/125] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:43 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:43.886790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:43.887551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:43.898171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:43.899516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:43.900262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:43.902692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:43.905317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:43.907290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:44.067160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:44.083763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e059e6bd-fde9-4b32-bbf6-748feb40b939 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1315 microversion: 2.1 time: 0.201191 Aug 24 10:03:44.084381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 59/126] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:43 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1315 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:44.540990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:44.542154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:44.568343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:44.569792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:44.570731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:44.571601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:44.573558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:44.574290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:44.758853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:44.760890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:44.775021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:44.775721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:44.776312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:44.777125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:44.777693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:44.778720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:44.837932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:44.841457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:44.847827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:44.849367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:44.850775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8affa5b0-2b55-458b-8363-c15e6853a98a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.314626 Aug 24 10:03:44.852081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 68/127] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:44.858909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-4b310add-94c7-4ca9-a143-5a9e155cea4e req-866e0997-c9ec-4465-89e6-c562f795240e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1", "name": "network-vif-plugged", "status": "completed", "tag": "1b9069bd-2630-4b65-9e91-d7695e3b6a8c"}, {"server_uuid": "d6a8cae5-8006-46a9-90b3-4556894041b0", "name": "network-vif-plugged", "status": "completed", "tag": "5930ef7d-e5d9-40c3-bc98-18c0ec1f1de4"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:44.871990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-4b310add-94c7-4ca9-a143-5a9e155cea4e req-866e0997-c9ec-4465-89e6-c562f795240e service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:44.872756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-4b310add-94c7-4ca9-a143-5a9e155cea4e req-866e0997-c9ec-4465-89e6-c562f795240e service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:44.873191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-4b310add-94c7-4ca9-a143-5a9e155cea4e req-866e0997-c9ec-4465-89e6-c562f795240e service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:44.888592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-4b310add-94c7-4ca9-a143-5a9e155cea4e req-866e0997-c9ec-4465-89e6-c562f795240e service nova] Creating event network-vif-plugged:1b9069bd-2630-4b65-9e91-d7695e3b6a8c for instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 on hv-cinder-85427 Aug 24 10:03:44.888592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-4b310add-94c7-4ca9-a143-5a9e155cea4e req-866e0997-c9ec-4465-89e6-c562f795240e service nova] Creating event network-vif-plugged:5930ef7d-e5d9-40c3-bc98-18c0ec1f1de4 for instance d6a8cae5-8006-46a9-90b3-4556894041b0 on hv-cinder-85427 Aug 24 10:03:44.894210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-4b310add-94c7-4ca9-a143-5a9e155cea4e req-866e0997-c9ec-4465-89e6-c562f795240e service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.040775 Aug 24 10:03:44.898102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 69/128] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:03:44 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:44.997660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:45.022138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f63a4350-58c5-4980-8e24-29dc8452b75a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.277882 Aug 24 10:03:45.036754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 60/129] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:44 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:45.105589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:45.106287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:45.113184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:45.113674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:45.114116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:45.114882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:45.115608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:45.116033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:45.271857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:45.280399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a87aa971-9783-4c7d-bd74-49b1066dfa5c tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1502 microversion: 2.1 time: 0.178872 Aug 24 10:03:45.286717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 70/130] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:45 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1502 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:45.874976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:45.875582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:45.881637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:45.882319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:45.882753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:45.883447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:45.883921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:45.884323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:46.018023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:46.026279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:46.027764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:46.028880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4e50a97c-3874-46b9-9aa9-d69f3415670f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.158296 Aug 24 10:03:46.029433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 61/131] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:45 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:46.043067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:46.043700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:46.049746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:46.050229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:46.050664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:46.051353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:46.051804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:46.052265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:46.185360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:46.193803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f6e95444-5367-4495-a8a1-06109ae2663c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.155053 Aug 24 10:03:46.194360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 71/132] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:46 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:47.050908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:47.051641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:47.057748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:47.058223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:47.058646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:47.059315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:47.059769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:47.060192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:47.195756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:47.205444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:47.206939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:47.209435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ca703439-f361-4f14-8762-ae142038cff8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.161098 Aug 24 10:03:47.209435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 62/133] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:47.216611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:47.217226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:47.224746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:47.225252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:47.225693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:47.226381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:47.226961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:47.227548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:47.363905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:47.371801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-92016951-d04e-4716-ab0e-945a5c86f6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.159565 Aug 24 10:03:47.372400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 72/134] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:47 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:48.232481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:48.234149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:48.240930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:48.241743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:48.242251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:48.243147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:48.248083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:48.248083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:48.397641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:48.399468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:48.410485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:48.410977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:48.411410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:48.412123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:48.412566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:48.412983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:48.423710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:48.433990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:48.436413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:48.438201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e89a6604-25df-4dfc-823d-24355194b8f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.211226 Aug 24 10:03:48.439931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 63/135] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:48 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:48.583702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:48.591960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c4150b8b-b4b4-42bb-9810-9ad0c3469431 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1322 microversion: 2.1 time: 0.201401 Aug 24 10:03:48.593244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 73/136] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:48 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1322 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:03:49.029564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b99aa248-b732-46b0-9eb5-424fd125c3f4 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ddf4efbb-c7db-4a94-8a6a-a8b691c44c65"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:49.033188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b99aa248-b732-46b0-9eb5-424fd125c3f4 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:49.058514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b99aa248-b732-46b0-9eb5-424fd125c3f4 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:49.060023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b99aa248-b732-46b0-9eb5-424fd125c3f4 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:49.060520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b99aa248-b732-46b0-9eb5-424fd125c3f4 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:49.061739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b99aa248-b732-46b0-9eb5-424fd125c3f4 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:49.062194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b99aa248-b732-46b0-9eb5-424fd125c3f4 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:49.062614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b99aa248-b732-46b0-9eb5-424fd125c3f4 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:49.463002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:49.463613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:49.470524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:49.471183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:49.471785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:49.472514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:49.473607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:49.474055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:49.667385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:49.677251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:49.678798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:49.679912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fe51c280-1dcc-4a39-a1a0-80fd90f26d13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1408 microversion: 2.20 time: 0.221094 Aug 24 10:03:49.680597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 74/137] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1408 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:03:49.697471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:49.698276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:49.716519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:49.717392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:49.718449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:49.719631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:49.720682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:49.721990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:49.901041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:49.910776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8ab7c58a-dec1-439d-9e86-a0e7299c5efc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1516 microversion: 2.1 time: 0.228331 Aug 24 10:03:49.911500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 75/138] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:49 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1516 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:49.933895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:49.936733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:49.944231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:49.944795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:49.945316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:49.946229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:49.946944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:49.947502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:50.059999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'flavor' on Instance uuid d6a8cae5-8006-46a9-90b3-4556894041b0 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:03:50.306681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e59893b5-8c26-4aae-945f-eabe710c97ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0/action" status: 200 len: 32 microversion: 2.1 time: 0.377292 Aug 24 10:03:50.313676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 76/139] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:03:49 2022] POST /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0/action => generated 32 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Aug 24 10:03:50.330037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b99aa248-b732-46b0-9eb5-424fd125c3f4 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "POST /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.307071 Aug 24 10:03:50.332278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 64/140] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:03:49 2022] POST /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d/os-volume_attachments => generated 194 bytes in 1312 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:50.357176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:50.359076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:50.374558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:50.375652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:50.377757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:50.384350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:50.384350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:50.384350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:50.647061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:50.658419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d5f3390a-bd7b-4fe3-95fb-2c56b2f1e5d1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.328039 Aug 24 10:03:50.659725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 77/141] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:50 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:50.704954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:50.705656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:50.715769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:50.716828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:50.717591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:50.719551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:50.720415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:50.721109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:50.974180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:50.983773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:50.986852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:50.988027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6005c6cd-d8c9-40a2-846c-633afc4cf5a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1600 microversion: 2.20 time: 0.287818 Aug 24 10:03:50.993269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 65/142] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:03:50 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1600 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:03:51.011289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6f7da635-29f9-48a5-929b-064c9e592d37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:51.012216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6f7da635-29f9-48a5-929b-064c9e592d37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:51.019190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f7da635-29f9-48a5-929b-064c9e592d37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:51.020014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f7da635-29f9-48a5-929b-064c9e592d37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:51.020709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f7da635-29f9-48a5-929b-064c9e592d37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:51.021660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f7da635-29f9-48a5-929b-064c9e592d37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:51.022326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f7da635-29f9-48a5-929b-064c9e592d37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:51.022951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f7da635-29f9-48a5-929b-064c9e592d37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:51.148740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6f7da635-29f9-48a5-929b-064c9e592d37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.141285 Aug 24 10:03:51.149649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 78/143] 10.222.0.86 () {60 vars in 1316 bytes} [Wed Aug 24 10:03:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-interface => generated 284 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:03:51.687289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:51.687969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:51.702196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:51.705936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:51.709264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:51.710242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:51.713455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:51.714688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:52.193516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:52.202830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4daa99ee-a8bc-4749-8a59-547fc532d26c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.522560 Aug 24 10:03:52.203396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 66/144] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:51 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:53.158778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-232be797-7e08-4513-b8fc-4902300f3cc9 req-08fa8afa-30b2-4328-a969-9d807cfd21a6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1", "tag": "1b9069bd-2630-4b65-9e91-d7695e3b6a8c"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:03:53.173667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-232be797-7e08-4513-b8fc-4902300f3cc9 req-08fa8afa-30b2-4328-a969-9d807cfd21a6 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:53.174423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-232be797-7e08-4513-b8fc-4902300f3cc9 req-08fa8afa-30b2-4328-a969-9d807cfd21a6 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:53.175568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-232be797-7e08-4513-b8fc-4902300f3cc9 req-08fa8afa-30b2-4328-a969-9d807cfd21a6 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:53.193699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-232be797-7e08-4513-b8fc-4902300f3cc9 req-08fa8afa-30b2-4328-a969-9d807cfd21a6 service nova] Creating event network-changed:1b9069bd-2630-4b65-9e91-d7695e3b6a8c for instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 on hv-cinder-85427 Aug 24 10:03:53.199353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-232be797-7e08-4513-b8fc-4902300f3cc9 req-08fa8afa-30b2-4328-a969-9d807cfd21a6 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045695 Aug 24 10:03:53.200698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 79/145] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:03:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:03:53.227194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:53.227814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:53.238919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:53.239410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:53.255975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:53.255975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:53.255975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:53.255975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:53.422405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:53.432204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a71db574-1d8b-4895-bb99-f6febb72803a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.208983 Aug 24 10:03:53.433613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 67/146] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:53 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:54.458913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:54.459870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:54.476432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:54.477455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:54.479459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:54.483462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:54.484649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:54.485541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:54.697712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:54.709569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2f1cc22e-0c65-4469-80b9-39d71d482be4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.255118 Aug 24 10:03:54.723608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 80/147] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:54 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:55.733749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:55.735247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:55.750022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:55.750758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:55.752629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:55.755360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:55.756936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:55.757615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:56.070642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:56.083015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0db002f3-1ff6-404a-a7cd-639eca7658aa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.355859 Aug 24 10:03:56.100236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 68/148] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:55 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:57.109154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:57.109154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:57.123380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:57.123995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:57.124402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:57.125937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:57.126417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:57.126846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:57.365551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:57.381480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4ba00b3b-7592-44e9-b335-c3b1a76e2d00 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.274341 Aug 24 10:03:57.382104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 81/149] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:57 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:58.300434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:58.300946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:58.319463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:58.320197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:58.321268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:58.322469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:58.323800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:58.324470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:58.363700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:03:58.421555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:58.422170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:58.436971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:58.439470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:58.440152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:58.446007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:58.446676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:58.447267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:58.488097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8047a2a5-de01-419f-b455-59c8de6222ca tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "DELETE /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 204 len: 0 microversion: 2.1 time: 0.191699 Aug 24 10:03:58.489959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 69/150] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:03:58 2022] DELETE /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:03:58.507232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:58.507879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:58.516980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:58.517479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:58.517926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:58.518632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:58.519084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:58.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:58.681730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:58.693558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a28d3f20-39e4-4752-847e-4f424b215e67 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.293849 Aug 24 10:03:58.694480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 82/151] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:58 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:03:58.756384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:58.774757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-14fd6e5b-0355-4dac-803b-eb4d47b625f0 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1554 microversion: 2.1 time: 0.278652 Aug 24 10:03:58.775873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 70/152] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:58 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1554 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:59.714720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:59.715316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:59.722367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:59.722848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:59.723270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:59.724151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:59.724699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:59.725163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:59.799655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:03:59.800270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:03:59.811727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:59.812922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:59.813872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:59.815888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:03:59.816601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:03:59.817402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:03:59.873812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:59.882891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-727b1c74-50eb-470c-bfe5-bcfec885c6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.171863 Aug 24 10:03:59.883566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 83/153] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:59 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:03:59.982114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:03:59.993067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a27291fb-2ffa-4408-811c-8b271d073ed3 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1554 microversion: 2.1 time: 0.199080 Aug 24 10:04:00.007470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 71/154] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:03:59 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1554 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:00.907061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:00.907683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:00.915173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:00.916078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:00.916650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:00.917424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:00.918099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:00.918594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:01.017010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:01.018501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:01.041070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:01.041910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:01.043070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:01.047441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:01.048841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:01.049359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:01.060526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:01.249159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4b8d531e382487a938546c009398b2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d455181e7e046f480437a8ab3620746', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:01.252921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:01.261364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a8ca3f26-2113-49dd-923d-0823f8682015 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 200 len: 1512 microversion: 2.1 time: 0.249406 Aug 24 10:04:01.262200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 72/155] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:01 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 1512 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:01.262802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6af9afe7-1166-4ec7-8fe4-0b6931ad1da2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.361266 Aug 24 10:04:01.264484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 84/156] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:00 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:01.296306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-c02a3158-23de-4e3e-8668-29baec27ef3e req-8ead1ce9-f0f2-4dcb-ab5d-7b3d94ec84ae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a4d2604d-01d1-4c1f-b02b-0c62ae2a027d", "name": "network-vif-deleted", "tag": "1a686d3f-cefc-4297-b3cd-7ebf8ce50b96"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:01.313974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-c02a3158-23de-4e3e-8668-29baec27ef3e req-8ead1ce9-f0f2-4dcb-ab5d-7b3d94ec84ae service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:01.314327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-c02a3158-23de-4e3e-8668-29baec27ef3e req-8ead1ce9-f0f2-4dcb-ab5d-7b3d94ec84ae service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:01.314935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-c02a3158-23de-4e3e-8668-29baec27ef3e req-8ead1ce9-f0f2-4dcb-ab5d-7b3d94ec84ae service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:01.333766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-c02a3158-23de-4e3e-8668-29baec27ef3e req-8ead1ce9-f0f2-4dcb-ab5d-7b3d94ec84ae service nova] Creating event network-vif-deleted:1a686d3f-cefc-4297-b3cd-7ebf8ce50b96 for instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d on hv-cinder-85427 Aug 24 10:04:01.339379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-c02a3158-23de-4e3e-8668-29baec27ef3e req-8ead1ce9-f0f2-4dcb-ab5d-7b3d94ec84ae service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065878 Aug 24 10:04:01.343873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 73/157] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:04:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:04:02.284533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:02.285263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:02.293630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:02.294563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:02.295286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:02.295536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:02.296747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:02.296857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:02.297496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:02.298133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:02.315299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:02.315802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:02.316279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:02.321223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:02.321717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:02.322315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:02.326610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:02.390779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] HTTP exception thrown: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Request method failure captured: Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d HTTP/1.1 Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdetGo5SZLNXd9ZUjHdGMuHa5WF2LlqqmTdgJiQnZvZgBeIbMq33UO74Y2VTsGF1WNCyh6x3Rw9iQNqOTfomErVs9ttdFwGmqgR2awpM1kcvlH1DgQpnlpjosd0ZNwQqOTangtboY2IxKt4fYMEs3zf-1UcvQKOGx3FQMC1JFpMsek Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 8d455181e7e046f480437a8ab3620746 Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:04:02.403506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/8d455181e7e046f480437a8ab3620746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/8d455181e7e046f480437a8ab3620746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_8d455181e7e046f480437a8ab3620746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/8d455181e7e046f480437a8ab3620746"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 8d455181e7e046f480437a8ab3620746 Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-DeleteServersTestJSON-1995826183-project Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: c4b8d531e382487a938546c009398b2e Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-DeleteServersTestJSON-1995826183-project Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.405752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': 'a4d2604d-01d1-4c1f-b02b-0c62ae2a027d'} Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:04:02.406687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.410752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.412250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Returning 404 to user: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:04:02.412250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-433202dd-3b6a-4805-8c3e-5aa8fd6670eb tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 404 len: 111 microversion: 2.1 time: 0.118085 Aug 24 10:04:02.421909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 74/158] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:02 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 111 bytes in 132 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:04:02.550801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:02.561728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:02.563876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:02.573754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6f18fc43-cc7b-4aeb-916a-ec72b22b568f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.293626 Aug 24 10:04:02.574803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:02.575908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:02.576823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:02.579323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:02.579838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:02.580535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:02.589220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 85/159] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:02 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:02.600135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] HTTP exception thrown: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Request method failure captured: Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d/os-volume_attachments HTTP/1.1 Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdetGo5SZLNXd9ZUjHdGMuHa5WF2LlqqmTdgJiQnZvZgBeIbMq33UO74Y2VTsGF1WNCyh6x3Rw9iQNqOTfomErVs9ttdFwGmqgR2awpM1kcvlH1DgQpnlpjosd0ZNwQqOTangtboY2IxKt4fYMEs3zf-1UcvQKOGx3FQMC1JFpMsek Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 8d455181e7e046f480437a8ab3620746 Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:04:02.602495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/8d455181e7e046f480437a8ab3620746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/8d455181e7e046f480437a8ab3620746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_8d455181e7e046f480437a8ab3620746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/8d455181e7e046f480437a8ab3620746"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 8d455181e7e046f480437a8ab3620746 Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-DeleteServersTestJSON-1995826183-project Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: c4b8d531e382487a938546c009398b2e Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-DeleteServersTestJSON-1995826183-project Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.603942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': 'a4d2604d-01d1-4c1f-b02b-0c62ae2a027d'} Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:04:02.604903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:02.606204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:02.609118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Returning 404 to user: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:04:02.611031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b8fb70a1-301f-4c20-9a78-207cc70e7eb5 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.053703 Aug 24 10:04:02.611969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 75/160] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:04:02 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d/os-volume_attachments => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:04:03.595836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:03.596536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:03.606533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:03.607535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:03.608294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:03.613811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:03.614583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:03.615233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:03.813684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:03.822560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dac4de95-a393-4991-be7b-ba525d31514b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.231372 Aug 24 10:04:03.823390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 86/161] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:03 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:03.844941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:03.844941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:03.851754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:03.852293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:03.852726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:03.854642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:03.855103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:03.855629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:03.870922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] HTTP exception thrown: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Request method failure captured: Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d HTTP/1.1 Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdetGo5SZLNXd9ZUjHdGMuHa5WF2LlqqmTdgJiQnZvZgBeIbMq33UO74Y2VTsGF1WNCyh6x3Rw9iQNqOTfomErVs9ttdFwGmqgR2awpM1kcvlH1DgQpnlpjosd0ZNwQqOTangtboY2IxKt4fYMEs3zf-1UcvQKOGx3FQMC1JFpMsek Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 8d455181e7e046f480437a8ab3620746 Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:04:03.872584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/8d455181e7e046f480437a8ab3620746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/8d455181e7e046f480437a8ab3620746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_8d455181e7e046f480437a8ab3620746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/8d455181e7e046f480437a8ab3620746"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 8d455181e7e046f480437a8ab3620746 Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-DeleteServersTestJSON-1995826183-project Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: c4b8d531e382487a938546c009398b2e Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-DeleteServersTestJSON-1995826183-project Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.874451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': 'a4d2604d-01d1-4c1f-b02b-0c62ae2a027d'} Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:04:03.875235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.876460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.878436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Returning 404 to user: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:04:03.878436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-06bdb026-76e7-4f51-851d-ebadb66c5617 tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "DELETE /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 404 len: 111 microversion: 2.1 time: 0.034518 Aug 24 10:04:03.899831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 76/162] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:04:03 2022] DELETE /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 111 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:04:03.908408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:03.909334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] [instance: a4d2604d-01d1-4c1f-b02b-0c62ae2a027d] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:03.917782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:03.918280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:03.918744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:03.919416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:03.919985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:03.924633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:03.943175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] HTTP exception thrown: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Request method failure captured: Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d HTTP/1.1 Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdetGo5SZLNXd9ZUjHdGMuHa5WF2LlqqmTdgJiQnZvZgBeIbMq33UO74Y2VTsGF1WNCyh6x3Rw9iQNqOTfomErVs9ttdFwGmqgR2awpM1kcvlH1DgQpnlpjosd0ZNwQqOTangtboY2IxKt4fYMEs3zf-1UcvQKOGx3FQMC1JFpMsek Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 8d455181e7e046f480437a8ab3620746 Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:04:03.949775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/8d455181e7e046f480437a8ab3620746"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/8d455181e7e046f480437a8ab3620746"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_8d455181e7e046f480437a8ab3620746"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/8d455181e7e046f480437a8ab3620746"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 8d455181e7e046f480437a8ab3620746 Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-DeleteServersTestJSON-1995826183 Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-DeleteServersTestJSON-1995826183-project Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: c4b8d531e382487a938546c009398b2e Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-DeleteServersTestJSON-1995826183-project Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.951712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': 'a4d2604d-01d1-4c1f-b02b-0c62ae2a027d'} Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:04:03.952498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. Aug 24 10:04:03.953577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:03.954536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] Returning 404 to user: Instance a4d2604d-01d1-4c1f-b02b-0c62ae2a027d could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:04:03.954536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f405c033-724a-422b-9699-3e1424c9b07d tempest-DeleteServersTestJSON-1995826183 tempest-DeleteServersTestJSON-1995826183-project] 10.222.0.86 "GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d" status: 404 len: 111 microversion: 2.1 time: 0.048219 Aug 24 10:04:03.956416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 87/163] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:03 2022] GET /compute/v2.1/servers/a4d2604d-01d1-4c1f-b02b-0c62ae2a027d => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:04:04.846797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:04.848330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:04.856383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:04.857807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:04.858575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:04.859928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:04.860562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:04.861204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:05.212100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:05.224178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f368d131-88b3-4eb0-8ad4-844ad98548a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.381372 Aug 24 10:04:05.225543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 77/164] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:04 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:06.251864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:06.254215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:06.267185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:06.267965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:06.268667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:06.271257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:06.272317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:06.273013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:06.611545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:06.622623 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-21764661-1386-4584-aab4-de008e5926f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.377965 Aug 24 10:04:06.626805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 88/165] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:06 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:07.652316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:07.654714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:07.662368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=64,nova_api:UPDATE=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:07.664710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:07.665513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:07.666159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:07.667454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:07.668325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:07.668956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:08.109630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:08.118112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b7ded2d4-3b6d-4908-af5c-6fd9df7b1ca0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.469986 Aug 24 10:04:08.119194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 78/166] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:07 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:09.139470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:09.140898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:09.143185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=60 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:09.153498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:09.154746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:09.155778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:09.168612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:09.172154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:09.181689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:09.507793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:09.520671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0eb81106-0c3e-4f78-afbd-cc616d54c4a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.384781 Aug 24 10:04:09.521257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 89/167] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:09 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:10.542368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:10.542969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:10.550406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:10.550877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:10.551570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:10.552363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:10.552875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:10.553492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:10.704008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:10.711901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-674e6941-b86b-470b-b634-51fd588b39d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.173283 Aug 24 10:04:10.713400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 79/168] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:10 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:11.734990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:11.735584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:11.749577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:11.750099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:11.750549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:11.751261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:11.755142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:11.755584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:11.946368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:11.956506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-58cf760f-2103-4182-a8ab-c3fd9843a6f4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.225647 Aug 24 10:04:11.958499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 90/169] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:11 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:12.981326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:12.982593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:12.999817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:13.000751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:13.001527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:13.002947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:13.004558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:13.005191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:13.430555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:13.443943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1d2be33d-ab8b-4f6f-99b6-d0a23b1547c1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.468651 Aug 24 10:04:13.444698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 80/170] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:12 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:14.465627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:14.466815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:14.473759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:14.478097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:14.478552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:14.481019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:14.482452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:14.483268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:14.838977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:14.889804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d667a20a-70af-4f9e-9e5b-9762673dc5ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.428213 Aug 24 10:04:14.891862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 91/171] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:14 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:15.927478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:15.936390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:15.957974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:15.958544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:15.958994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:15.959711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:15.960201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:15.960630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:16.549580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:16.563886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d2183a2c-bdca-4223-95ec-ad1dfc84521a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.640481 Aug 24 10:04:16.566073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 81/172] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:15 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:17.588816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:17.589433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:17.625630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:17.627300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:17.628931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:17.631767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:17.633780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:17.634456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:18.113794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:18.147810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-62b79f35-4506-4874-94cb-693bacab2f02 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.560230 Aug 24 10:04:18.151510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 92/173] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:17 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:19.174341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:19.175615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:19.187066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:19.189102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:19.189805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:19.190547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:19.191008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:19.191436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:19.383410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fd879d4a-657c-4651-9569-d5ef692701d5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'create', calling method: .version_select at 0x7fa33db304c0>, body: {"keypair": {"name": "tempest-keypair-1630910318", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDN+MBoaQ0bYJ2xqNKwVq6xz90E2t3fmCcCRLYD+d8144Tgo08mm+xV7fbwuq9o0hvHkrzHHSQQtqeTKYtv39QUJOXVCYHNIyfJKPO3OM0hTrU3TyGsZ5nFC29fMqmbGfw=="}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:19.406941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-fd879d4a-657c-4651-9569-d5ef692701d5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Getting quotas for project e055ba4967db42108c069ebc3843dcf6. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:04:19.424911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-fd879d4a-657c-4651-9569-d5ef692701d5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Getting quotas for user 1de97cce55e944dbb62a920ea11734f5 and project e055ba4967db42108c069ebc3843dcf6. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:04:19.474017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fd879d4a-657c-4651-9569-d5ef692701d5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.776355 Aug 24 10:04:19.497827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 82/174] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:04:18 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 802 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:04:19.940094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:19.948388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-07d49d39-28aa-42d4-b85a-3bd12c08a793 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.778538 Aug 24 10:04:19.952593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 93/175] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:19 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 783 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:20.980531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:20.982949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:21.002015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:21.010232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:21.010689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:21.011431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:21.011886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:21.016168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:21.506659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:21.517851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-17037175-c459-49ba-89f8-48243d56cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.545781 Aug 24 10:04:21.518441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 83/176] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:20 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:22.539502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:22.540804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:22.553504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:22.554005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:22.557394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:22.558378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:22.559213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:22.559614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:22.749383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:22.758405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f6d2ea6b-2d82-4d0a-990d-82c9ddd8a31b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.222967 Aug 24 10:04:22.759494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 94/177] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:22 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:23.166757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-528483959", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "65642c38-aa5e-4b7b-9606-eb1b26338156"}], "security_groups": [{"name": "tempest-securitygroup--803329791"}], "key_name": "tempest-keypair-1630910318", "user_data": "}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:23.379644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] validate_networks() for [('65642c38-aa5e-4b7b-9606-eb1b26338156', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:04:23.796749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:23.797367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:23.810041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:23.810835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:23.812527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:23.814811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:23.819217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:23.819968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:24.365545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:24.376348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5bfa74a8-ca2f-4e4f-b77a-29bdd0c6ea80 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1522 microversion: 2.1 time: 0.596636 Aug 24 10:04:24.392442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 95/178] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:23 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1522 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:24.517047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:04:24.518053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:04:24.518730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:04:24.519370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:04:24.519981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:04:24.539625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:24.540146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:24.540608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:24.545474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:24.546020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:24.546693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:24.585615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Getting quotas for project e055ba4967db42108c069ebc3843dcf6. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:04:24.603222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Getting quotas for user 1de97cce55e944dbb62a920ea11734f5 and project e055ba4967db42108c069ebc3843dcf6. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:04:24.637952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:04:24.649815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:04:24.759682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-164bb9bf-9af7-4301-96b0-d234b5acd872 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.597102 Aug 24 10:04:24.774846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-427cc09c-166b-4be8-8185-23781c803aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:24.775855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-427cc09c-166b-4be8-8185-23781c803aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:24.776918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 84/179] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:04:23 2022] POST /compute/v2.1/servers => generated 401 bytes in 1615 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:04:25.155812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-427cc09c-166b-4be8-8185-23781c803aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:25.173481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-427cc09c-166b-4be8-8185-23781c803aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1237 microversion: 2.1 time: 0.402967 Aug 24 10:04:25.181327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 96/180] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:24 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1237 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:25.401009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:25.401779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:25.416331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:25.420675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:25.424017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:25.425378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:25.429924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:25.432543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:25.860503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:25.877662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-96f888bd-42c2-433b-b3dd-62387e2ebd1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1502 microversion: 2.1 time: 0.480296 Aug 24 10:04:25.884305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 85/181] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:25 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1502 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:25.904515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:25.905136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:25.915657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:25.916526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:25.918740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:25.920613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:25.921601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:25.922275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:26.204677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:26.205337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:26.222973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:26.223668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:26.225400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:26.228098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:26.229597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:26.232607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:26.394585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:26.415543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-263948dc-c452-43d0-9255-4c441d36af8b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1233 microversion: 2.1 time: 0.216486 Aug 24 10:04:26.431992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 86/182] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:26 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1233 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:26.776433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:26.796514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0d8ba0c5-862f-4c98-9363-10dc3a7e25db tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1515 microversion: 2.1 time: 0.897118 Aug 24 10:04:26.800461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 97/183] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:25 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1515 bytes in 902 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:27.438983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:27.439587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:27.459998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:27.461885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:27.465685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:27.468389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:27.468888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:27.471700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:27.863752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:27.882712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-810fb2b8-342f-49e8-9a2e-fefc8a6b5727 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1295 microversion: 2.1 time: 0.445909 Aug 24 10:04:27.896557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 87/184] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:27 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1295 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:28.822758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:28.826373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:28.834525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:28.835492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:28.836464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:28.850591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:28.852240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:28.853397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:28.919779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:28.920322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:28.926253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'flavor' on Instance uuid 7ab73a23-282f-4f54-9a62-5a417d7e7603 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:04:28.933320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:28.937721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:28.938358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:28.943275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:28.944563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:28.947019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:29.127117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:29.135048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1be48178-3145-4abf-b232-22217bd7f38c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1295 microversion: 2.1 time: 0.219772 Aug 24 10:04:29.135687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 88/185] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:28 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1295 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:29.189920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1bce135f-0ae9-4816-8658-2e2458e368cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603/action" status: 200 len: 32 microversion: 2.1 time: 0.378226 Aug 24 10:04:29.191294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 98/186] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:04:28 2022] POST /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603/action => generated 32 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 24 10:04:29.208646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:29.210087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:29.237323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:29.237829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:29.257366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:29.273275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:29.275276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:29.288812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:29.833252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:29.846556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-abfa784d-bab9-4017-8369-1266662fbfbf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.649958 Aug 24 10:04:29.847720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 89/187] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:29 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:30.158317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:30.159676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:30.170920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:30.176655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:30.181425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:30.182396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:30.183146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:30.184307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:30.337111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:30.348876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b66dd922-c58b-4262-92a4-aa468cbdc660 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1295 microversion: 2.1 time: 0.194585 Aug 24 10:04:30.349808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 99/188] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:30 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1295 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:30.868353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:30.868889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:30.888087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:30.888598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:30.889038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:30.889762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:30.890194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:30.890617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:31.167083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:31.169330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=85 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:31.184458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ffb78547-87fa-4224-b952-632c6faef248 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.319597 Aug 24 10:04:31.186448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 90/189] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:30 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:31.370142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:31.372399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:31.378996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:31.379757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:31.380285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:31.380897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:31.388964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:31.389491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:31.717453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:31.747068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fcf4f8ed-6cd6-4698-9436-ea5d624c81da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1362 microversion: 2.1 time: 0.380720 Aug 24 10:04:31.749729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 100/190] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:31 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1362 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:32.027299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-2ff37b9e-186c-4c9f-93be-606bb0e48910 req-9f449ebd-761f-4e3e-8c67-e4cb20e8aac8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5692cc07-3f16-47a4-9993-503b451d9951", "tag": "c35480cd-41a8-4a27-93df-3ded8cf5eeda"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:32.049267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-2ff37b9e-186c-4c9f-93be-606bb0e48910 req-9f449ebd-761f-4e3e-8c67-e4cb20e8aac8 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:32.049752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-2ff37b9e-186c-4c9f-93be-606bb0e48910 req-9f449ebd-761f-4e3e-8c67-e4cb20e8aac8 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:32.050259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-2ff37b9e-186c-4c9f-93be-606bb0e48910 req-9f449ebd-761f-4e3e-8c67-e4cb20e8aac8 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:32.074934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-2ff37b9e-186c-4c9f-93be-606bb0e48910 req-9f449ebd-761f-4e3e-8c67-e4cb20e8aac8 service nova] Creating event network-changed:c35480cd-41a8-4a27-93df-3ded8cf5eeda for instance 5692cc07-3f16-47a4-9993-503b451d9951 on hv-cinder-85427 Aug 24 10:04:32.080563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-2ff37b9e-186c-4c9f-93be-606bb0e48910 req-9f449ebd-761f-4e3e-8c67-e4cb20e8aac8 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059574 Aug 24 10:04:32.082528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 91/191] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:04:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:04:32.211072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1634266093", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "networks": [{"uuid": "fa3862ad-9d16-4db7-987f-c546a6024a5d"}]}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:32.213234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:32.214048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:32.227058 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:32.227633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:32.229320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:32.230133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:32.232085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:32.232901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:32.271394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:32.314250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] validate_networks() for [('fa3862ad-9d16-4db7-987f-c546a6024a5d', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:04:32.495019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:32.503627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3e3defa7-5aff-4c3d-a39e-bcad4d2384e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.300535 Aug 24 10:04:32.522566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 101/192] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:32 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:32.770130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:32.770916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:32.787168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:32.792481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:32.794676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:32.802748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:32.804191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:32.805719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:33.015268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:33.027623 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0c38c563-f836-4793-afde-67445d0b038c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1362 microversion: 2.1 time: 0.261608 Aug 24 10:04:33.048863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 102/193] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:32 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1362 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:33.436464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:04:33.437427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:04:33.438242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:04:33.438654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:04:33.439061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:04:33.448914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:33.449393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:33.449995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:33.451043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:33.451674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:33.452112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:33.478999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Getting quotas for project f12553f79dbd4d75a291d3a07a8dbbba. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:04:33.488811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Getting quotas for user 731cf6df5bda42148e1420e365306445 and project f12553f79dbd4d75a291d3a07a8dbbba. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:04:33.504475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:04:33.507212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:04:33.530066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:33.531303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:33.540689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:33.541172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:33.541778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:33.543460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:33.544227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:33.545018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:33.572984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-424981a4-55a4-4250-8f74-449ca3dd6976 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.366075 Aug 24 10:04:33.574788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 92/194] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:04:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 1369 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:04:33.587683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d510b56f-e768-4472-aaa5-1a18a1ba872d tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:33.588665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d510b56f-e768-4472-aaa5-1a18a1ba872d tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:33.681150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d510b56f-e768-4472-aaa5-1a18a1ba872d tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:33.690485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d510b56f-e768-4472-aaa5-1a18a1ba872d tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1214 microversion: 2.1 time: 0.108628 Aug 24 10:04:33.690845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 93/195] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:33 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1214 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:33.755416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:33.769826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9eaead39-4241-4836-b5fa-5e824a8aa4b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.244375 Aug 24 10:04:33.781460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 103/196] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:33 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:34.050868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:34.051535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:34.087156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:34.087666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:34.088135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:34.088861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:34.089830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:34.092146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:34.551248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:34.562226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2b887aee-5481-4a7b-8528-6931b18d1d02 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1362 microversion: 2.1 time: 0.515877 Aug 24 10:04:34.564106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 94/197] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:34 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1362 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:34.713264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:34.713878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:34.724142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:34.724631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:34.726602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:34.727486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:34.736016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:34.737750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:34.790684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:34.791804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:34.799029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:34.801366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:34.802253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:34.803288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:34.803953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:34.805617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:34.851456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:34.860826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-21d290c6-25e9-470f-bb9f-1977d24d879b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1266 microversion: 2.1 time: 0.151356 Aug 24 10:04:34.862005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 104/198] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:34 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:34.975424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:34.994678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-00539ee9-2e63-483c-a2f1-4d3f0d0cf3b2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.207875 Aug 24 10:04:34.997506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 95/199] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:34 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:35.120409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d9a43504-e69e-481e-8189-96df3019b6f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Action: 'create', calling method: .version_select at 0x7fa33e68d8b0>, body: {"keypair": {"name": "tempest-TestShelveInstance-2092334647", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLiARa5mAGQzmSedg7rA9Ch6LYX9CFpz6VtEtB9bjaCDu3NprABbGfDVOooQnoHOwwjIR3Z45qEqCQ1yeJBHI5+nyBWHOxXPVBsk0KEKuFHy3Ke8TZ//lC2mZTZ2qtohxA=="}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:35.150765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-d9a43504-e69e-481e-8189-96df3019b6f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Getting quotas for project 302c5ab2e660436e8a0af25c12ceee20. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:04:35.155634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-d9a43504-e69e-481e-8189-96df3019b6f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Getting quotas for user 1c665588c50d47dbaf1f8fde082ee73a and project 302c5ab2e660436e8a0af25c12ceee20. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:04:35.235242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d9a43504-e69e-481e-8189-96df3019b6f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.371829 Aug 24 10:04:35.243640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 105/200] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:04:34 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:04:35.588792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:35.590657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:35.597649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:35.598140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:35.600713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:35.607405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:35.610730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:35.611437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:35.884638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:35.885863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:35.914408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:35.915342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:35.917280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:35.922593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:35.923608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:35.925515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:36.063241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:36.082328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9ebc5a2a-d4ee-4bb0-9de1-7a6f2ef48912 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1362 microversion: 2.1 time: 0.496414 Aug 24 10:04:36.085156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 96/201] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:35 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1362 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:36.094341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:36.101937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:36.110967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:36.111952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:36.112640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:36.119838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:36.121337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:36.121795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:36.413016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:36.421047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b4098139-9c6f-4a3a-9cbc-388f00666cae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.334654 Aug 24 10:04:36.421460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:36.429576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-15bb0b51-c5eb-4668-8ed4-cc2289c51bf3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1272 microversion: 2.1 time: 0.549989 Aug 24 10:04:36.430156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 106/202] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:35 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1272 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:36.436325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 97/203] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:36 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:37.109300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:37.110004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:37.129356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:37.131740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:37.132937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:37.138072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:37.138557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:37.139196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:37.308769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-07b4be3c-1c3b-42d0-9823-fbadd6934ca5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:37.310674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-07b4be3c-1c3b-42d0-9823-fbadd6934ca5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:37.326697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-07b4be3c-1c3b-42d0-9823-fbadd6934ca5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:37.329396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-07b4be3c-1c3b-42d0-9823-fbadd6934ca5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:37.331872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-07b4be3c-1c3b-42d0-9823-fbadd6934ca5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:37.334379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-07b4be3c-1c3b-42d0-9823-fbadd6934ca5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:37.335640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-07b4be3c-1c3b-42d0-9823-fbadd6934ca5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:37.338682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-07b4be3c-1c3b-42d0-9823-fbadd6934ca5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:37.410136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-07b4be3c-1c3b-42d0-9823-fbadd6934ca5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/ips" status: 200 len: 152 microversion: 2.20 time: 0.104541 Aug 24 10:04:37.410740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 98/204] 10.222.0.86 () {60 vars in 1289 bytes} [Wed Aug 24 10:04:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/ips => generated 152 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:04:37.434515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-664b36a2-98b6-43af-bae1-bf2693f3a33e req-0dd532eb-d024-452e-a6fc-607d119b2c6a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5692cc07-3f16-47a4-9993-503b451d9951", "name": "network-vif-plugged", "status": "completed", "tag": "c35480cd-41a8-4a27-93df-3ded8cf5eeda"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:37.469578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-664b36a2-98b6-43af-bae1-bf2693f3a33e req-0dd532eb-d024-452e-a6fc-607d119b2c6a service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:37.471242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-664b36a2-98b6-43af-bae1-bf2693f3a33e req-0dd532eb-d024-452e-a6fc-607d119b2c6a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:37.474411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-664b36a2-98b6-43af-bae1-bf2693f3a33e req-0dd532eb-d024-452e-a6fc-607d119b2c6a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:37.515305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-664b36a2-98b6-43af-bae1-bf2693f3a33e req-0dd532eb-d024-452e-a6fc-607d119b2c6a service nova] Creating event network-vif-plugged:c35480cd-41a8-4a27-93df-3ded8cf5eeda for instance 5692cc07-3f16-47a4-9993-503b451d9951 on hv-cinder-85427 Aug 24 10:04:37.532651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-664b36a2-98b6-43af-bae1-bf2693f3a33e req-0dd532eb-d024-452e-a6fc-607d119b2c6a service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.111954 Aug 24 10:04:37.533568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 99/205] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:04:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:04:37.539195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:37.540288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:37.547476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:37.548373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:37.549092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:37.550048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:37.550727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:37.551417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:37.674119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:37.689054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4d20e46d-8f04-4690-90a2-82a3c53b5aa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1362 microversion: 2.1 time: 0.586759 Aug 24 10:04:37.701695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 107/206] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:37 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1362 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:37.706275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:37.709778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:37.744995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:37.746389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:37.749684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:37.750686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:37.753951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:37.757251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:37.870229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:37.882575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-297d74a4-2dc4-4cb3-939d-ef29683ff03e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.344605 Aug 24 10:04:37.884771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 100/207] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:37 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:38.264884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:38.279577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9caba7de-9dbd-4164-b573-db31559a4b12 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1314 microversion: 2.1 time: 0.583052 Aug 24 10:04:38.291227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 108/208] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:37 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1314 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:38.434917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-5289b890-64c5-4c60-ae35-6457491ade23 req-84d67e24-9eea-446a-8253-121c9deff782 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1186b839-48f8-4d68-8bc5-c2138d5fa990", "tag": "c91f9278-9595-4eb0-947a-3d1f16231006"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:38.446248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=7 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:38.452700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5289b890-64c5-4c60-ae35-6457491ade23 req-84d67e24-9eea-446a-8253-121c9deff782 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:38.453382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5289b890-64c5-4c60-ae35-6457491ade23 req-84d67e24-9eea-446a-8253-121c9deff782 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:38.454619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5289b890-64c5-4c60-ae35-6457491ade23 req-84d67e24-9eea-446a-8253-121c9deff782 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:38.470379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-5289b890-64c5-4c60-ae35-6457491ade23 req-84d67e24-9eea-446a-8253-121c9deff782 service nova] Creating event network-changed:c91f9278-9595-4eb0-947a-3d1f16231006 for instance 1186b839-48f8-4d68-8bc5-c2138d5fa990 on hv-cinder-85427 Aug 24 10:04:38.478954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-5289b890-64c5-4c60-ae35-6457491ade23 req-84d67e24-9eea-446a-8253-121c9deff782 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048295 Aug 24 10:04:38.480715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 101/209] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:04:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:04:38.710199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:38.710809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:38.717613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:38.719530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:38.720987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:38.724729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:38.725305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:38.725772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:38.906807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:38.908011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:38.915209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:38.915841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:38.916472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:38.917931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:38.918494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:38.919024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:38.923193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:38.931324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dc426e74-6ac0-4a97-aa19-3d5faad23179 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1362 microversion: 2.1 time: 0.224945 Aug 24 10:04:38.932083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 109/210] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:38 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1362 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:39.073075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:39.081162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a0a1fa39-f188-4fdb-8c11-6a954397b2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.178910 Aug 24 10:04:39.081848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 102/211] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:38 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:39.314611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:39.315316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:39.317121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:39.323652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:39.324183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:39.324649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:39.325348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:39.325798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:39.326212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:39.570019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:39.581167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f653bc93-7381-4530-b580-d7fdb437f370 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1314 microversion: 2.1 time: 0.270422 Aug 24 10:04:39.596284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 110/212] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:39 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1314 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:39.953454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:39.954067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:39.960896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:39.961553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:39.961990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:39.962687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:39.963163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:39.963823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.103039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:40.103657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:40.110383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.111765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.112479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.113402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.114051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.114472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.146411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:40.160166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dcfc5a1b-79cb-45e0-9638-cc1c4493cc5a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1549 microversion: 2.1 time: 0.209217 Aug 24 10:04:40.161510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 103/213] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:39 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1549 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:40.203263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6c5cc1dc-7684-4b1c-9ca3-5d287685f961 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:40.204210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6c5cc1dc-7684-4b1c-9ca3-5d287685f961 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:40.211609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c5cc1dc-7684-4b1c-9ca3-5d287685f961 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.213178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c5cc1dc-7684-4b1c-9ca3-5d287685f961 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.213808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c5cc1dc-7684-4b1c-9ca3-5d287685f961 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.214789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c5cc1dc-7684-4b1c-9ca3-5d287685f961 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.215428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c5cc1dc-7684-4b1c-9ca3-5d287685f961 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.216034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c5cc1dc-7684-4b1c-9ca3-5d287685f961 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.286912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:40.296009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7f602aff-5e45-4547-ae42-3fcb1c2e88eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.196764 Aug 24 10:04:40.296826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 111/214] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:40 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:40.307315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6c5cc1dc-7684-4b1c-9ca3-5d287685f961 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.112072 Aug 24 10:04:40.308538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 104/215] 10.222.0.86 () {58 vars in 1275 bytes} [Wed Aug 24 10:04:40 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-interface => generated 285 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:04:40.574808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:40.575487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:40.581354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.581832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.582285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.582966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.583416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.583828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.603396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:40.604010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:40.613397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.614174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.614746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.615583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.616195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.616764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.619280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lazy-loading 'flavor' on Instance uuid 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:04:40.889043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:40.904850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-44887766-b917-4cf2-a1bb-3591a918c2a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1314 microversion: 2.1 time: 0.304272 Aug 24 10:04:40.906580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 105/216] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:40 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1314 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:40.941529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9a0adb9d-b772-4733-ac78-7b4c6ffe2584 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "POST /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/action" status: 202 len: 0 microversion: 2.20 time: 0.370672 Aug 24 10:04:40.942147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 112/217] 10.222.0.86 () {62 vars in 1319 bytes} [Wed Aug 24 10:04:40 2022] POST /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/action => generated 0 bytes in 372 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:04:40.952256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:40.953229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:40.961592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.962805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.963627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:40.965497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:40.966622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:40.967348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:41.228982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:41.239852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:41.241537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:41.243017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-26683a54-4d46-4b30-a7c3-e3c6ee53be0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.296243 Aug 24 10:04:41.244109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 106/218] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:41.320072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:41.321084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:41.330000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:41.330860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:41.331458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:41.332667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:41.333119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:41.333697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:41.567101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:41.567296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-8ecbbd1b-bb87-4707-b106-bbce88bcadf2 req-bff849ad-2123-4f08-946c-cfd7588f3f39 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5692cc07-3f16-47a4-9993-503b451d9951", "tag": "c35480cd-41a8-4a27-93df-3ded8cf5eeda"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:41.576311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a8a07ffb-9946-4a3b-aec9-8e5679a10b6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.261504 Aug 24 10:04:41.577016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 113/219] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:41 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:41.601715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-8ecbbd1b-bb87-4707-b106-bbce88bcadf2 req-bff849ad-2123-4f08-946c-cfd7588f3f39 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:41.602177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-8ecbbd1b-bb87-4707-b106-bbce88bcadf2 req-bff849ad-2123-4f08-946c-cfd7588f3f39 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:41.602597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-8ecbbd1b-bb87-4707-b106-bbce88bcadf2 req-bff849ad-2123-4f08-946c-cfd7588f3f39 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:41.615572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-8ecbbd1b-bb87-4707-b106-bbce88bcadf2 req-bff849ad-2123-4f08-946c-cfd7588f3f39 service nova] Creating event network-changed:c35480cd-41a8-4a27-93df-3ded8cf5eeda for instance 5692cc07-3f16-47a4-9993-503b451d9951 on hv-cinder-85427 Aug 24 10:04:41.623461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-8ecbbd1b-bb87-4707-b106-bbce88bcadf2 req-bff849ad-2123-4f08-946c-cfd7588f3f39 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061943 Aug 24 10:04:41.624813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 107/220] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:04:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:04:41.927980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:41.928643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:41.935961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:41.940739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:41.941295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:41.942219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:41.942889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:41.943419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:42.269960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:42.271457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:42.285181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:42.288076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:42.291325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:42.292096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:42.294272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:42.296973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:42.298980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:42.301321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9ad44acc-c98b-45a3-90ce-3e5aaea39473 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1314 microversion: 2.1 time: 0.377205 Aug 24 10:04:42.302258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 114/221] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:41 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1314 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:42.528832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:42.539125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:42.540797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:42.542620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3219cf86-cfce-48bb-af10-563fedb10c35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.278054 Aug 24 10:04:42.556070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 108/222] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:42 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:04:42.598798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:42.599968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:42.609707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:42.610375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:42.610873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:42.611846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:42.612502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:42.613253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:42.816341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:42.825234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dac6f2ac-cc4d-4dc8-9031-da6061ada1c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.230814 Aug 24 10:04:42.839856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 115/223] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:42 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:43.323937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:43.324726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:43.330866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:43.331458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:43.331990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:43.332800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:43.333356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:43.334018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:43.461859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:43.478424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:43.486245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-28f4e355-5f33-4149-9701-7e46c0cfe4ee tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1314 microversion: 2.1 time: 0.167022 Aug 24 10:04:43.486804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 109/224] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:43 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:43.567208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:43.567807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:43.576438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:43.576983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:43.577424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:43.578126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:43.578582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:43.579007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:43.717328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:43.726200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:43.728184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:43.729415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c1f50122-88d1-4ea6-a195-03c67720a57b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.166635 Aug 24 10:04:43.730153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 116/225] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:43.847398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:43.848013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:43.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=18 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:43.855492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:43.855976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:43.856433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:43.857122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:43.857568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:43.858237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:44.002477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:44.013484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8f87dec9-8af7-4812-9927-2dd6423d8d18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.169824 Aug 24 10:04:44.014081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 110/226] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:43 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:44.507150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:44.507869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:44.513772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:44.514256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:44.514695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:44.515388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:44.515836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:44.516285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:44.520363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:44.684407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:44.693549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-815b53a9-5d6e-467c-a7f7-211815c19634 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1314 microversion: 2.1 time: 0.190018 Aug 24 10:04:44.694691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 117/227] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:44 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1314 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:44.757123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:44.757648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:44.764540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:44.765029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:44.765463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:44.766151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:44.766606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:44.767030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:44.899586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:44.908445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:44.909955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:44.911184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5892c166-8bb6-4188-882f-b2f776635035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.158258 Aug 24 10:04:44.911767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 111/228] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:45.033418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:45.033943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:45.039922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:45.040448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:45.040876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:45.041551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:45.042001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:45.042419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:45.175746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=6 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:45.180114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:45.188995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7f35b77f-df37-4902-8bc8-4e994c5abfb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.159316 Aug 24 10:04:45.189568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 118/229] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:45 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:45.231255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-b6622e42-8159-4a5e-9d93-5d9ebce6d936 req-ad4bf781-8b58-49dd-b749-8b2bc7a6c3fb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1186b839-48f8-4d68-8bc5-c2138d5fa990", "name": "network-vif-plugged", "status": "completed", "tag": "c91f9278-9595-4eb0-947a-3d1f16231006"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:45.241388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-b6622e42-8159-4a5e-9d93-5d9ebce6d936 req-ad4bf781-8b58-49dd-b749-8b2bc7a6c3fb service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:45.241878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-b6622e42-8159-4a5e-9d93-5d9ebce6d936 req-ad4bf781-8b58-49dd-b749-8b2bc7a6c3fb service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:45.242393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-b6622e42-8159-4a5e-9d93-5d9ebce6d936 req-ad4bf781-8b58-49dd-b749-8b2bc7a6c3fb service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:45.254668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-b6622e42-8159-4a5e-9d93-5d9ebce6d936 req-ad4bf781-8b58-49dd-b749-8b2bc7a6c3fb service nova] Creating event network-vif-plugged:c91f9278-9595-4eb0-947a-3d1f16231006 for instance 1186b839-48f8-4d68-8bc5-c2138d5fa990 on hv-cinder-85427 Aug 24 10:04:45.259719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-b6622e42-8159-4a5e-9d93-5d9ebce6d936 req-ad4bf781-8b58-49dd-b749-8b2bc7a6c3fb service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032126 Aug 24 10:04:45.261055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 112/230] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:04:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:04:45.719321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:45.721638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:45.740596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:45.741276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:45.741713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:45.742427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:45.743118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:45.743531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:45.935417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:45.936763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:45.943046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:45.943741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:45.944209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:45.945093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:45.945714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:45.946120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:45.951086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:45.975216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:45.983500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-89208c09-4024-484d-b634-194d5d4aa611 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1314 microversion: 2.1 time: 0.271762 Aug 24 10:04:45.984106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 119/231] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:45 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1314 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:46.212833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:46.214957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:46.225142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:46.226703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:46.227757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:46.229650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:46.231642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:46.232382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:46.401761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:46.411883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:46.414587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:46.416199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-16e052d4-48e9-4493-8ac2-58674c7e837f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.485970 Aug 24 10:04:46.438089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 113/232] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:45 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:04:46.580454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:46.589617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-58b4c820-6064-4467-93d3-b6e53bf4f16d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.382682 Aug 24 10:04:46.590742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 120/233] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:46 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:47.004986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:47.005596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:47.012540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:47.013033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:47.013471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:47.014168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:47.014623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:47.015047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:47.159573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:47.168139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-da86c349-071c-475a-ac8f-86358c83e885 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1498 microversion: 2.1 time: 0.166878 Aug 24 10:04:47.180557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 114/234] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:46 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1498 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:47.444716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:47.445744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:47.452556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:47.453117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:47.453632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:47.454397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:47.455087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:47.455868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:47.614395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:47.615284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:47.622478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:47.623229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:47.623798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:47.624724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:47.625264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:47.625624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:47.656138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:47.683198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:47.686040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:47.688037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-867c03f1-5277-4611-bffc-1789ce04de33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.247545 Aug 24 10:04:47.688948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 121/235] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:47.783345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:47.790988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-571c8831-87c5-4f6d-8e48-86fe2ebf4e58 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.183511 Aug 24 10:04:47.792102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 115/236] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:47 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:48.710699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:48.711295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:48.718068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:48.718555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:48.718986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:48.719678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:48.720144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:48.720591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:48.827375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:48.828184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:48.843949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:48.848095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:48.849174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:48.850395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:48.852344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:48.853339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:48.946773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:48.957605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:48.960102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:48.960929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-99b1ef4e-6cf6-46d1-961a-289aaa44c826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.254064 Aug 24 10:04:48.961688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 122/237] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:48 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:49.051543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:49.059782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4433e344-faf2-4766-a592-7869daf39953 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.240412 Aug 24 10:04:49.060388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 116/238] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:48 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:49.983476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:49.984100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:49.998083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:49.998083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:49.998083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:49.998083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:49.998381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:49.998381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:50.082920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:50.083726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:50.091161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:50.091729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:50.092273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:50.093107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:50.093698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:50.094529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:50.153896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:50.162079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:50.164377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:50.164985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b2815b1f-50dc-41c1-837e-061824a3fe52 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.186066 Aug 24 10:04:50.165537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 123/239] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:50.248502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:50.256376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-293e1c50-7ab8-4912-99b9-58dae047c017 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.178269 Aug 24 10:04:50.257023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 117/240] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:50 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:51.188602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:51.189189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:51.195044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:51.195522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:51.195976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:51.196680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:51.197136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:51.197582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:51.266806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:51.266806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:51.273582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:51.274068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:51.274534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:51.275247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:51.276277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:51.276704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:51.303556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:04:51.341744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:51.352893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:51.355053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:51.356253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-571a54ed-858b-4a95-b901-60f8a4157246 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.171474 Aug 24 10:04:51.356791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 124/241] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:51.361679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:51.362288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:51.368381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:51.368862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:51.369289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:51.370139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:51.370715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:51.371153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:51.386725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cf3619b7-90f2-4e25-a2dc-75aff27cb22c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "DELETE /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 204 len: 0 microversion: 2.1 time: 0.124210 Aug 24 10:04:51.388070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 118/242] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:04:51 2022] DELETE /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:04:51.398204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:51.399135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:51.406610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:51.407095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:51.407531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:51.408269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:51.408736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:51.409148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:51.507602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:51.515934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f37c0c4d-55aa-4296-8f7e-48c65390d58c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.158034 Aug 24 10:04:51.516533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 125/243] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:51 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:51.548591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:51.556405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-baf807f1-7af1-4599-b989-6295450c8051 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1504 microversion: 2.1 time: 0.163540 Aug 24 10:04:51.557047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 119/244] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:51 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1504 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:51.784769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-110506565", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:51.841109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] validate_networks() for None {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:04:52.133752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:04:52.134198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:04:52.134667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:04:52.135074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:04:52.135495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:04:52.144475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:52.144956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:52.145387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:52.146106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:52.146557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:52.146974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:52.164329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Getting quotas for project b3d697163a8d4ee197f1b6b2856f08ae. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:04:52.169051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Getting quotas for user 2063027ade874ab19c7df1655ebd6b85 and project b3d697163a8d4ee197f1b6b2856f08ae. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:04:52.179856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:04:52.182344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:04:52.228735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7ddcbaa2-4e2d-4fba-8eeb-fb1aab1281ca tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.448059 Aug 24 10:04:52.229401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 126/245] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:04:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 450 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:04:52.241237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1941a329-3981-4f04-ac12-39b0e8ff6e33 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:52.241845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1941a329-3981-4f04-ac12-39b0e8ff6e33 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:52.295357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1941a329-3981-4f04-ac12-39b0e8ff6e33 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:52.302746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1941a329-3981-4f04-ac12-39b0e8ff6e33 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1214 microversion: 2.1 time: 0.065292 Aug 24 10:04:52.303314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 120/246] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:52 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1214 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:52.378592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:52.379201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:52.385309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:52.386502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:52.387103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:52.387800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:52.388294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:52.388720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:52.528192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:52.537622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:52.538020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:52.538252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:52.540226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:52.542823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-34df6ae7-20d2-4ecd-8c85-86388bb8efef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.167881 Aug 24 10:04:52.544327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 127/247] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:52 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:52.545709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:52.547432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:52.548134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:52.549151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:52.549812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:52.550360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:52.579869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:52.580706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:52.590193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:52.591004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:52.591702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:52.592893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:52.594022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:52.594609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:52.715092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:52.725761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e16561d7-736f-4f39-8a67-f87576912d17 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.192247 Aug 24 10:04:52.726506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 121/248] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:52 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:52.754276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:52.762346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6f1dc378-4d55-4d8c-81bc-a27ededf8b54 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1504 microversion: 2.1 time: 0.187146 Aug 24 10:04:52.762913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 128/249] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:52 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1504 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:53.325617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:53.326333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:53.332794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.333280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.333722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.334462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.334914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.335331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.398866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:53.406698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-292fc85d-da88-4914-afa3-52497c127669 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1210 microversion: 2.1 time: 0.085899 Aug 24 10:04:53.407268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 122/250] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:53 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:53.567502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:53.568157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:53.575005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.575498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.575932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.576675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.577132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.577556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.748868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:53.749493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:53.756437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.756989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.757447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.758507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.758969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.759399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.799981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:53.812569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:53.814193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:53.815464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0c10981b-f7dd-41bd-bd1d-cebb63930ac4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.252020 Aug 24 10:04:53.816475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 129/251] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:53 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:53.821505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:53.822338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:53.831775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.832372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.833105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.833892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.834433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.834860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.915991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:53.924988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e1b7e233-ec5a-4c00-9dcd-9bc7bd2af3b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1501 microversion: 2.1 time: 0.180797 Aug 24 10:04:53.925654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 123/252] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:53 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1501 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:53.949246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "62516296-49f4-41c6-8b29-4f1f9f210a31"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:53.952476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:53.958405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.959055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.960007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.960583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:53.961127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:53.961532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:53.992680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Cannot 'attach_volume' instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 while it is in vm_state rescued Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: POST /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603/os-volume_attachments HTTP/1.1 Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 74 Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:04:53.994367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {"volumeAttachment": {"volumeId": "62516296-49f4-41c6-8b29-4f1f9f210a31"}} Aug 24 10:04:53.995976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Cannot 'attach_volume' instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 while it is in vm_state rescued Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '7ab73a23-282f-4f54-9a62-5a417d7e7603', 'body': {'volumeAttachment': {'volumeId': '62516296-49f4-41c6-8b29-4f1f9f210a31'}}} Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 in vm_state rescued. Cannot attach_volume while the instance is in this state. Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:04:54.000739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 while it is in vm_state rescued Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:54.002129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:54.003251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:04:54.003251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:04:54.003251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 while it is in vm_state rescued Aug 24 10:04:54.003251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:04:54.003251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 409 to user: Cannot 'attach_volume' instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 while it is in vm_state rescued {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:04:54.003251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c6f76cf8-c177-4bc3-84c4-f034d98dce53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.051314 Aug 24 10:04:54.003251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 124/253] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:04:53 2022] POST /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603/os-volume_attachments => generated 152 bytes in 53 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Aug 24 10:04:54.005783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d6b37221-da54-4582-af41-5b9e38b1a20e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:54.006320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d6b37221-da54-4582-af41-5b9e38b1a20e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:54.012988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6b37221-da54-4582-af41-5b9e38b1a20e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:54.013520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6b37221-da54-4582-af41-5b9e38b1a20e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:54.013958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6b37221-da54-4582-af41-5b9e38b1a20e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:54.014811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6b37221-da54-4582-af41-5b9e38b1a20e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:54.015290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6b37221-da54-4582-af41-5b9e38b1a20e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:54.015713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6b37221-da54-4582-af41-5b9e38b1a20e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:54.062366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:54.070156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0c38b119-1327-456b-b2e6-e3bd1e63e3a3 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 200 len: 1504 microversion: 2.1 time: 0.252409 Aug 24 10:04:54.070987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 130/254] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:53 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 1504 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:54.079724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-02d19025-176b-449e-b75f-10b5821454dc req-769b0141-b1a1-45f3-a255-a41559f8d264 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1186b839-48f8-4d68-8bc5-c2138d5fa990", "name": "network-vif-deleted", "tag": "c91f9278-9595-4eb0-947a-3d1f16231006"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:54.092068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-02d19025-176b-449e-b75f-10b5821454dc req-769b0141-b1a1-45f3-a255-a41559f8d264 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:54.092799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-02d19025-176b-449e-b75f-10b5821454dc req-769b0141-b1a1-45f3-a255-a41559f8d264 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:54.093414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-02d19025-176b-449e-b75f-10b5821454dc req-769b0141-b1a1-45f3-a255-a41559f8d264 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:54.109208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-02d19025-176b-449e-b75f-10b5821454dc req-769b0141-b1a1-45f3-a255-a41559f8d264 service nova] Creating event network-vif-deleted:c91f9278-9595-4eb0-947a-3d1f16231006 for instance 1186b839-48f8-4d68-8bc5-c2138d5fa990 on hv-cinder-85427 Aug 24 10:04:54.112492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d6b37221-da54-4582-af41-5b9e38b1a20e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603/action" status: 202 len: 0 microversion: 2.1 time: 0.111282 Aug 24 10:04:54.113987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 125/255] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:04:53 2022] POST /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603/action => generated 0 bytes in 114 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 24 10:04:54.114323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-02d19025-176b-449e-b75f-10b5821454dc req-769b0141-b1a1-45f3-a255-a41559f8d264 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041807 Aug 24 10:04:54.115196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 131/256] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:04:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:04:54.123483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:54.124191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:54.130349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:54.130823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:54.131250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:54.131961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:54.132391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:54.132805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:54.293495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:54.306847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bcda0ef0-9e22-4a54-829e-2e193e424b40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1509 microversion: 2.1 time: 0.187609 Aug 24 10:04:54.307978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 126/257] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:54 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1509 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:54.428919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:54.429750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:54.439632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:54.439632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:54.439632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:54.439632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:54.440106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:54.440106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:54.570442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:54.581281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-22ddc0ca-f2f9-4165-9ba4-2008e724e435 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1266 microversion: 2.1 time: 0.156604 Aug 24 10:04:54.582437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 132/258] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:54 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:54.840830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:54.841443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:54.862873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:54.863468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:54.864103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:54.865368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:54.865891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:54.866321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:55.076402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:55.085924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:55.087916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:55.089897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e01dcbd4-d8ee-400d-9f19-e163532e7929 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.254677 Aug 24 10:04:55.090875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 127/259] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:54 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:55.098225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:55.098829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 1186b839-48f8-4d68-8bc5-c2138d5fa990] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:55.105230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:55.105874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:55.106663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:55.107960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:55.108266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:55.108855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:55.122815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] HTTP exception thrown: Instance 1186b839-48f8-4d68-8bc5-c2138d5fa990 could not be found. Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Request method failure captured: Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 HTTP/1.1 Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfeuW7pI2kYhP44XP-MRTEr2eD_1700H-cXU5l5eTLTqIds9t2Vild7swpbEX9UaBuyYBTb5bUQWIpHUTB_4D8rbSKC9K6B7UBBhrL6E7uzgysDpSq29VGhp0voj-JrQPskFR0xvvYCDujfAkV-Cuds76ITbaRkNre1k_lJbj_1kAKs Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: f12553f79dbd4d75a291d3a07a8dbbba Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesActionsTest-1093348380 Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:04:55.124787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f12553f79dbd4d75a291d3a07a8dbbba"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f12553f79dbd4d75a291d3a07a8dbbba"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f12553f79dbd4d75a291d3a07a8dbbba"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f12553f79dbd4d75a291d3a07a8dbbba"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesActionsTest-1093348380 Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: f12553f79dbd4d75a291d3a07a8dbbba Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesActionsTest-1093348380 Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesActionsTest-1093348380-project Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 731cf6df5bda42148e1420e365306445 Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesActionsTest-1093348380-project Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 1186b839-48f8-4d68-8bc5-c2138d5fa990 could not be found. Aug 24 10:04:55.126143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '1186b839-48f8-4d68-8bc5-c2138d5fa990'} Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1186b839-48f8-4d68-8bc5-c2138d5fa990 could not be found. Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:04:55.128720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1186b839-48f8-4d68-8bc5-c2138d5fa990 could not be found. Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1186b839-48f8-4d68-8bc5-c2138d5fa990 could not be found. Aug 24 10:04:55.130115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:04:55.131368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Returning 404 to user: Instance 1186b839-48f8-4d68-8bc5-c2138d5fa990 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:04:55.131368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1692c8e4-ad29-4184-b67f-51276c4ade9b tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990" status: 404 len: 111 microversion: 2.1 time: 0.036313 Aug 24 10:04:55.131368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 133/260] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:55 2022] GET /compute/v2.1/servers/1186b839-48f8-4d68-8bc5-c2138d5fa990 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:04:55.239713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-967893611", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "networks": [{"uuid": "fa3862ad-9d16-4db7-987f-c546a6024a5d"}]}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:04:55.297436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] validate_networks() for [('fa3862ad-9d16-4db7-987f-c546a6024a5d', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:04:55.335240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:55.335936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:55.343255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:55.343925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:55.344399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:55.345841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:55.346390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:55.346808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:55.489620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:55.500780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-952b8cda-df44-4866-b554-736b58d5fb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1509 microversion: 2.1 time: 0.174109 Aug 24 10:04:55.501358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 134/261] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:55 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1509 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:55.602113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:55.602755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:55.608798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:55.609284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:55.609717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:55.610444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:55.610876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:55.611403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:55.685748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:55.691297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:55.696458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-254d7613-6ec9-4f64-8a96-80a18e4ba4c9 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1272 microversion: 2.1 time: 0.098120 Aug 24 10:04:55.697033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 135/262] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:55 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1272 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:55.865199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:04:55.868911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:04:55.869578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:04:55.870887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:04:55.871259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:04:55.884634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:55.885155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:55.885591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:55.886318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:55.886769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:55.887371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:55.906147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Getting quotas for project f12553f79dbd4d75a291d3a07a8dbbba. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:04:55.910727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Getting quotas for user 731cf6df5bda42148e1420e365306445 and project f12553f79dbd4d75a291d3a07a8dbbba. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:04:55.941440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:04:55.943784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:04:56.005093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4fbb6d83-4096-4dd9-9b12-c75f297821fe tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.769082 Aug 24 10:04:56.006264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 128/263] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:04:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 771 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:04:56.017260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a4ea4be7-4278-45cd-862a-2c65dfe5bfe8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:56.018002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a4ea4be7-4278-45cd-862a-2c65dfe5bfe8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:56.086608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a4ea4be7-4278-45cd-862a-2c65dfe5bfe8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:56.094666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a4ea4be7-4278-45cd-862a-2c65dfe5bfe8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1213 microversion: 2.1 time: 0.081306 Aug 24 10:04:56.096259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 136/264] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:56 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1213 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:04:56.115076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:56.115675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:56.129484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:56.130084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:56.130609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:56.131405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:56.132294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:56.132939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:56.138179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:56.446549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:56.458482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:56.460933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:56.462475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-58cbc4c5-d07c-4eaf-8c7b-3b10887c9935 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.351171 Aug 24 10:04:56.464604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 129/265] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:56.522711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:56.523357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:56.530374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:56.530861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:56.531331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:56.532109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:56.532601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:56.533028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:56.672535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:56.680292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-071e7a29-216e-4b8b-ad8e-425156691e29 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1509 microversion: 2.1 time: 0.161829 Aug 24 10:04:56.693664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 137/266] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:56 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1509 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:56.719649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:56.720290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:56.726277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:56.726760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:56.727193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:56.727878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:56.728353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:56.728775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:56.792613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:56.801578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6cd40a8c-38c8-46b4-a638-a10566ba0b8a tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1272 microversion: 2.1 time: 0.085911 Aug 24 10:04:56.802280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 130/267] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:56 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1272 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:57.117308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:57.118097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:57.124970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:57.125392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:57.125826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:57.126515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:57.126975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:57.127738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:57.207875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:57.215716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7dd2dba0-f1bd-4819-befb-be7926021171 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1217 microversion: 2.1 time: 0.102704 Aug 24 10:04:57.216550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 138/268] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:57 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1217 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:57.488615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:57.489130 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:57.496981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:57.497678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:57.498101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:57.499812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:57.500684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:57.501431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:57.647700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:57.658284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:57.660301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:57.661631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c7327297-781e-4ebb-a0d6-4955f2c647a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.177639 Aug 24 10:04:57.662406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 131/269] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:57 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:57.701612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:57.702736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:57.709324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:57.709907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:57.710563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:57.711271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:57.711723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:57.712321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:57.823691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:57.824867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:57.836172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:57.837236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:57.838031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:57.839728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:57.840692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:57.841724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:57.925179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:57.933752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ba8119e8-74dc-4102-b472-cfba07999e60 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1272 microversion: 2.1 time: 0.114827 Aug 24 10:04:57.940290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 132/270] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:57 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1272 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:58.238520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:58.239194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:58.248277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:58.252270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:58.253460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:58.253460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:58.253900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:58.256320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:58.267597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:58.278880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d1bf2d69-3889-49a0-bb69-45a987fa4258 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1509 microversion: 2.1 time: 0.580992 Aug 24 10:04:58.280414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 139/271] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:57 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1509 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:58.337827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:58.347461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-162eb0e2-675a-48a0-b79f-b21a82bff0b5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1265 microversion: 2.1 time: 0.114055 Aug 24 10:04:58.348371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 133/272] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:58 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1265 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:58.685238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:58.685950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:58.696883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:58.698964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:58.699641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:58.700545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:58.703372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:58.703970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:58.955275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:58.955889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:58.967194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:58.968025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:58.968685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:58.969726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:58.970388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:58.970994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:59.091725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:59.103873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3414c34a-507a-46d9-b012-423eef53b82b tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1272 microversion: 2.1 time: 0.152337 Aug 24 10:04:59.105550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 134/273] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:58 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1272 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:59.189092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:59.212680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:59.215029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:59.216300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d6f402ea-2847-4bff-a5d7-66acd2a7516b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.534973 Aug 24 10:04:59.217375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 140/274] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:04:58 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:04:59.303937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:59.304787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:59.314102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:59.314867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:59.315690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:59.316654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:59.318635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:59.319558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:59.373462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:04:59.374466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:04:59.387609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:59.388366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:59.389098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:59.390193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:04:59.391718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:04:59.392332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:04:59.533553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:59.560003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9546ad6e-6900-4252-9067-f17e4efc0123 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1271 microversion: 2.1 time: 0.193434 Aug 24 10:04:59.561317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 141/275] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:59 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1271 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:04:59.654077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:04:59.669616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b19b2911-d7c0-469e-ac73-487fda5a8f6e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1509 microversion: 2.1 time: 0.371884 Aug 24 10:04:59.672307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 135/276] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:04:59 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1509 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:00.126396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:00.127578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:00.134023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:00.135030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:00.135748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:00.136505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:00.137333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:00.137765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:00.243390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:00.243998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:00.251164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:00.251839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:00.252463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:00.253351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:00.253966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:00.254589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:00.411882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:00.423154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:00.424715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:00.425910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1634c437-173b-47bc-9635-2e82e8e6251b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.186452 Aug 24 10:05:00.426466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 136/277] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:00 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:00.451963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:00.459885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-54fd4da5-c124-49a2-91c2-630e4562fcc3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.337590 Aug 24 10:05:00.460574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 142/278] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:00 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:00.581652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:00.582176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:00.589332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:00.589524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:00.589966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:00.590657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:00.591108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:00.591533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:00.694842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:00.695446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:00.703077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:00.703847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:00.704494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:00.705342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:00.705944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:00.706627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:00.731056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:00.739459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-093599a6-9211-4fcf-bc99-9cad383ae7aa tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1313 microversion: 2.1 time: 0.161681 Aug 24 10:05:00.740063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 137/279] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:00 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:01.001011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:01.011811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4657dac0-244a-4eab-af85-b2acaf56dd30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1509 microversion: 2.1 time: 0.320771 Aug 24 10:05:01.012977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 143/280] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:00 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1509 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:05:01.450740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:01.451540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:01.464712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:01.465555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:01.466254 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:01.467165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:01.468570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:01.476947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:01.482007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:01.482864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:01.489020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:01.493879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:01.496260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:01.497377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:01.497995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:01.498681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:01.698832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:01.710692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:01.712568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:01.714327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1f908d19-18fe-4e24-aee5-7105ffd17166 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.268565 Aug 24 10:05:01.732653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 138/281] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:01.737825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-dbdbd0d9-3f18-4b6c-8004-3a8bd4451e06 req-20e9d1ef-e6d8-4ccb-ab2c-5ba43f8cce14 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "47745245-e791-4bdd-b31d-6f745be13913", "tag": "860d5021-b87f-44d1-9b8a-9c8005fd4363"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:01.749514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-dbdbd0d9-3f18-4b6c-8004-3a8bd4451e06 req-20e9d1ef-e6d8-4ccb-ab2c-5ba43f8cce14 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:01.750314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-dbdbd0d9-3f18-4b6c-8004-3a8bd4451e06 req-20e9d1ef-e6d8-4ccb-ab2c-5ba43f8cce14 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:01.750983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-dbdbd0d9-3f18-4b6c-8004-3a8bd4451e06 req-20e9d1ef-e6d8-4ccb-ab2c-5ba43f8cce14 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:01.773926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-dbdbd0d9-3f18-4b6c-8004-3a8bd4451e06 req-20e9d1ef-e6d8-4ccb-ab2c-5ba43f8cce14 service nova] Creating event network-changed:860d5021-b87f-44d1-9b8a-9c8005fd4363 for instance 47745245-e791-4bdd-b31d-6f745be13913 on hv-cinder-85427 Aug 24 10:05:01.780804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-dbdbd0d9-3f18-4b6c-8004-3a8bd4451e06 req-20e9d1ef-e6d8-4ccb-ab2c-5ba43f8cce14 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046517 Aug 24 10:05:01.781968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 139/282] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:01.795358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:01.796453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:01.799423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:01.805645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:01.806482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:01.807697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:01.808692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:01.809611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:01.810307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:02.027174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:02.055628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9e96b7ee-7734-4445-b9a9-7591ee530843 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.577831 Aug 24 10:05:02.056906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 144/283] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:01 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:02.062935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:02.063789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:02.071127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:02.071889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:02.072576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:02.073432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:02.074998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:02.075585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:02.129221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:02.140991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6264d4bd-2094-4578-9ec8-7ef5172f794c tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1313 microversion: 2.1 time: 0.351298 Aug 24 10:05:02.141704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 140/284] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:01 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1313 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:02.154008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:02.314667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-e5bfd3ce-f6be-4d48-9d57-188e6858f2ca req-1c0c1659-79e0-4b05-a7fb-08a4a69526b5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173", "tag": "990aec14-7426-422b-af96-eba82ff28e09"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:02.335266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-e5bfd3ce-f6be-4d48-9d57-188e6858f2ca req-1c0c1659-79e0-4b05-a7fb-08a4a69526b5 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:02.335750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-e5bfd3ce-f6be-4d48-9d57-188e6858f2ca req-1c0c1659-79e0-4b05-a7fb-08a4a69526b5 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:02.336644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-e5bfd3ce-f6be-4d48-9d57-188e6858f2ca req-1c0c1659-79e0-4b05-a7fb-08a4a69526b5 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:02.350559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-e5bfd3ce-f6be-4d48-9d57-188e6858f2ca req-1c0c1659-79e0-4b05-a7fb-08a4a69526b5 service nova] Creating event network-changed:990aec14-7426-422b-af96-eba82ff28e09 for instance 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 on hv-cinder-85427 Aug 24 10:05:02.356381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-e5bfd3ce-f6be-4d48-9d57-188e6858f2ca req-1c0c1659-79e0-4b05-a7fb-08a4a69526b5 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047860 Aug 24 10:05:02.358257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 141/285] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:05:02.724067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:02.746617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:02.747875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:02.753866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8c0a4985-ea22-48dc-8432-d794a10aeecd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1515 microversion: 2.1 time: 0.694427 Aug 24 10:05:02.757910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 145/286] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:02 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1515 bytes in 700 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:02.758917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:02.759612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:02.760337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:02.761461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:02.762107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:02.762717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:02.869200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-550493860", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-2092334647", "security_groups": [{"name": "tempest-secgroup-smoke-1405369785"}], "networks": [{"uuid": "2babaa12-dbf7-4c7e-af99-1b3a3b9f418e"}], "block_device_mapping_v2": [{"uuid": "24511da4-d0ea-4ef8-8d0c-075ba0bc5d3d", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:03.029747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:03.040581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:03.043544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:03.045059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-46d87f0c-98e0-4cb9-aff9-17da209c4af5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1725 microversion: 2.20 time: 0.309791 Aug 24 10:05:03.045830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 142/287] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:02 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1725 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:03.083948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:03.084767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:03.093626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:03.094075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:03.095201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:03.096518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:03.098896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:03.099886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:03.241471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] validate_networks() for [('2babaa12-dbf7-4c7e-af99-1b3a3b9f418e', None, None, None, None, None)] {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:05:03.397528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:03.406148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c454abe6-f020-4d81-829b-901e9dbc2530 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.328054 Aug 24 10:05:03.406860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 143/288] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:03 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:03.414482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:03.415092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:03.423568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:03.424367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:03.425510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:03.426481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:03.427096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:03.427716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:03.647737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:03.660838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-46047469-5bd5-48aa-a01f-9bd7be832da9 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1313 microversion: 2.1 time: 0.252664 Aug 24 10:05:03.662526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 144/289] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:03 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1313 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:04.069915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:04.071819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:04.084379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:04.085572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:04.086633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:04.088065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:04.089579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:04.090218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:04.343095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:05:04.343823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:04.344474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:05:04.345188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:05:04.345945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:05:04.346575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:05:04.352300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:04.354378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:04.355574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ee2bd9c1-c9fb-4e74-927a-07664007da1c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1746 microversion: 2.20 time: 0.290915 Aug 24 10:05:04.363944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:04.364922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 145/290] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1746 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:04.368812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:04.372147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:04.372983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:04.373437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:04.373865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:04.373994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e608bae3-f241-44e7-bd36-8edd63934df2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'create', calling method: .version_select at 0x7fa33ddcc3a0>, body: {"keypair": {"name": "tempest-keypair-1354646386", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMGxsGFe9jCreHTVyFAaClQ1otGbhlqyKe8g+j5ZHiJSNq69lC+93iFJZS+QLPVUYIeLl9bo0XxhgTvyxINPHUUFWAktGKsfUuCGXqGNqTaicLP0oPQKNmNov1Mou4z3Ng=="}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:04.408580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-e608bae3-f241-44e7-bd36-8edd63934df2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Getting quotas for project 1bdfd8ab42f9496d87a8cfcf78d7edc3. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:05:04.416691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-e608bae3-f241-44e7-bd36-8edd63934df2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Getting quotas for user 46c01f8b99e24be5b6c2d73b9b1bd9b8 and project 1bdfd8ab42f9496d87a8cfcf78d7edc3. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:05:04.435461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Getting quotas for project 302c5ab2e660436e8a0af25c12ceee20. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:05:04.444442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Getting quotas for user 1c665588c50d47dbaf1f8fde082ee73a and project 302c5ab2e660436e8a0af25c12ceee20. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:05:04.450683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e608bae3-f241-44e7-bd36-8edd63934df2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.084515 Aug 24 10:05:04.451424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 146/291] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:05:04 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:04.457168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:04.458519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:04.466080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:04.466684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:04.467577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:04.468530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:04.469120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:04.469667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:04.470886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:04.574755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:05:04.577005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='24511da4-d0ea-4ef8-8d0c-075ba0bc5d3d',volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:05:04.849310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2b0ee3c9-13b3-46c3-9627-d96674559f72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.984619 Aug 24 10:05:04.856086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 146/292] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:05:02 2022] POST /compute/v2.1/servers => generated 406 bytes in 1992 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 24 10:05:04.862686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:04.863300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:04.870248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:04.870987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:04.871830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:04.872736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:04.873352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:04.873777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:05.008944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:05.025312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-09ad888f-9cc3-4a3b-be76-021324192f74 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.572481 Aug 24 10:05:05.026837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 147/293] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:04 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:05.041332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-21a92d46-a640-4bca-8f56-47a66d2db7a3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:05.042932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-21a92d46-a640-4bca-8f56-47a66d2db7a3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:05.043260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:05.066150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1c5adddb-373f-4f9a-89e0-747998baf215 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1313 microversion: 2.1 time: 0.205356 Aug 24 10:05:05.066794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 147/294] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:04 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1313 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:05.393358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:05.394190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:05.407464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:05.408353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:05.408797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:05.412280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:05.412949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:05.416139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:05.420002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-21a92d46-a640-4bca-8f56-47a66d2db7a3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:05.429652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-21a92d46-a640-4bca-8f56-47a66d2db7a3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1088 microversion: 2.1 time: 0.399613 Aug 24 10:05:05.430531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 148/295] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1088 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:05.778219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:05.788224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:05.793763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:05.795155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ed9acb19-3258-4593-8cb1-8f5d0b4648e7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1746 microversion: 2.20 time: 0.407165 Aug 24 10:05:05.811883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 148/296] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1746 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:06.049421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:06.050352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:06.062606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:06.073974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:06.074518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:06.075481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:06.076264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:06.076728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:06.077235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:06.088987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:06.089736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:06.098065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:06.104269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:06.104704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:06.105152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:06.108698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:06.109216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:06.109657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:06.543863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:06.557897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-17c04d75-dd50-4b2c-8736-07663e8e8401 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1313 microversion: 2.1 time: 0.473966 Aug 24 10:05:06.572124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 149/297] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:06 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1313 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:06.578214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:06.595456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:06.601720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:06.608623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-544714af-104c-42be-9181-767fff88e979 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.560143 Aug 24 10:05:06.614022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 149/298] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:06 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:06.625838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:06.628387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:06.629028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:06.633278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:06.635363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:06.637869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:06.819488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:06.820404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:06.844929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:06.845716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:06.846106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:06.846823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:06.850385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:06.850385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:07.277042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:07.302772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fc94dc41-de92-403c-ab47-1040f448ed17 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1186 microversion: 2.1 time: 0.726157 Aug 24 10:05:07.304621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 150/299] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:06 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1186 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:07.579269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:07.581483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:07.593636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:07.594409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:07.596632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:07.597763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:07.598461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:07.599419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:07.843967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:07.854611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:07.856462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:07.874974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dcbfa517-4d01-44d8-a5f8-c5528db86631 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 1.060699 Aug 24 10:05:07.891860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 150/300] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 1078 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:07.911685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:07.912560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:07.919121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:07.920327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:07.921080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:07.931668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:07.932551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:07.933198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:08.402152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:08.425112 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a99fd5ad-b06c-4d9f-badb-2d171ec563c5 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1313 microversion: 2.1 time: 0.849525 Aug 24 10:05:08.427411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 151/301] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:07 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1313 bytes in 853 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:05:08.446287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:08.447362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:08.464645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:08.466219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:08.466938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:08.467943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:08.469319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:08.470314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:08.641555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:08.653696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3f210cbb-3876-4841-817f-7e08f3b40044 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1192 microversion: 2.1 time: 0.222850 Aug 24 10:05:08.656029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 152/302] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:08 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1192 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:08.908173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:08.912303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:08.928710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:08.930369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:08.931080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:08.937295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:08.937977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:08.938837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:09.449068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:09.457915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:09.472459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:09.477013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8cda9e87-a7d6-4005-b799-33129abfb779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.574651 Aug 24 10:05:09.485059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 153/303] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:08 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:09.497387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-f390df9b-834d-4c55-a721-54908df39fe2 req-ca761346-6fcf-494f-b02a-e251c932bfa2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1", "name": "network-vif-plugged", "status": "completed", "tag": "1b9069bd-2630-4b65-9e91-d7695e3b6a8c"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:09.503117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:09.518052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-571156c1-65d5-4e55-b521-b6d292e073e0 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 1.614915 Aug 24 10:05:09.518897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 151/304] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:07 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 1627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:09.526868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:09.533023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:09.549543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:09.550222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:09.550771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:09.551594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:09.552236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:09.552908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:09.554544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-f390df9b-834d-4c55-a721-54908df39fe2 req-ca761346-6fcf-494f-b02a-e251c932bfa2 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:09.555056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-f390df9b-834d-4c55-a721-54908df39fe2 req-ca761346-6fcf-494f-b02a-e251c932bfa2 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:09.555502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-f390df9b-834d-4c55-a721-54908df39fe2 req-ca761346-6fcf-494f-b02a-e251c932bfa2 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:09.570527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:09.598304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-f390df9b-834d-4c55-a721-54908df39fe2 req-ca761346-6fcf-494f-b02a-e251c932bfa2 service nova] Creating event network-vif-plugged:1b9069bd-2630-4b65-9e91-d7695e3b6a8c for instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 on hv-cinder-85427 Aug 24 10:05:09.609693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-f390df9b-834d-4c55-a721-54908df39fe2 req-ca761346-6fcf-494f-b02a-e251c932bfa2 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.122859 Aug 24 10:05:09.611545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 154/305] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:09.679184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:09.681190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:09.704299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:09.705946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:09.707731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:09.709751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:09.710916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:09.712008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:10.218133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:10.231050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dabbfdee-188f-47d8-ae7c-84a0487e185f tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1313 microversion: 2.1 time: 0.710991 Aug 24 10:05:10.246622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 152/306] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:09 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1313 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:10.371813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:10.386029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c08658ec-5b9a-4cd2-9137-70bb9bb0ecae tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.710143 Aug 24 10:05:10.386840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 155/307] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:09 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:05:10.519098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:10.521756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:10.554105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:10.555207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:10.561870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:10.564358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:10.567712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:10.568287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:10.568742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:10.570421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:10.571408 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:10.571779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:10.573042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:10.573608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:10.574715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:10.580834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:10.969319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:10.980233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:10.982830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:10.984480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-972a70a4-4f3c-4531-845d-0c78aa04dd37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.472293 Aug 24 10:05:10.997801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 153/308] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:10 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:11.080820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1380759207", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3742113f-8471-4780-be0c-20ea54885f07"}], "security_groups": [{"name": "tempest-securitygroup--1823014725"}], "key_name": "tempest-keypair-1354646386", "user_data": "}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:11.215458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:11.227194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9f0d028c-6dfa-4cde-a1c8-6c526a00c97c tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.683429 Aug 24 10:05:11.227696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 156/309] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:10 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 685 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:11.259310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:11.259975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:11.271282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:11.271785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:11.272250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] validate_networks() for [('3742113f-8471-4780-be0c-20ea54885f07', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:05:11.272400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:11.272964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:11.276754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:11.277504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:12.025531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:05:12.026088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:05:12.029599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:05:12.030148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:05:12.030672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:05:12.049092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:12.051710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:12.052455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:12.055715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:12.057097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:12.059740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:12.085595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Getting quotas for project 1bdfd8ab42f9496d87a8cfcf78d7edc3. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:05:12.093122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Getting quotas for user 46c01f8b99e24be5b6c2d73b9b1bd9b8 and project 1bdfd8ab42f9496d87a8cfcf78d7edc3. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:05:12.110932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:05:12.113618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:05:12.124593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:12.132958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bf184a96-6c4b-4301-ad92-a8d7396e3ccd tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1313 microversion: 2.1 time: 0.879163 Aug 24 10:05:12.139667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 157/310] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:11 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1313 bytes in 887 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:12.147565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:12.160105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:12.178866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:12.182242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:12.182856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:12.183112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-63d860cf-ffe2-4447-b1b4-c6e0eb2011c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.106180 Aug 24 10:05:12.184010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:12.185182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 154/311] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:05:11 2022] POST /compute/v2.1/servers => generated 409 bytes in 1111 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:05:12.186073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:12.188290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:12.194600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-0b611996-7846-494f-95ee-747f400db699 req-a426372e-ae75-48db-8dc5-eb42cddb378f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "47745245-e791-4bdd-b31d-6f745be13913", "name": "network-vif-plugged", "status": "completed", "tag": "860d5021-b87f-44d1-9b8a-9c8005fd4363"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:12.236357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-0b611996-7846-494f-95ee-747f400db699 req-a426372e-ae75-48db-8dc5-eb42cddb378f service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:12.236857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-0b611996-7846-494f-95ee-747f400db699 req-a426372e-ae75-48db-8dc5-eb42cddb378f service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:12.237297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-0b611996-7846-494f-95ee-747f400db699 req-a426372e-ae75-48db-8dc5-eb42cddb378f service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:12.269631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-0b611996-7846-494f-95ee-747f400db699 req-a426372e-ae75-48db-8dc5-eb42cddb378f service nova] Creating event network-vif-plugged:860d5021-b87f-44d1-9b8a-9c8005fd4363 for instance 47745245-e791-4bdd-b31d-6f745be13913 on hv-cinder-85427 Aug 24 10:05:12.280500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-0b611996-7846-494f-95ee-747f400db699 req-a426372e-ae75-48db-8dc5-eb42cddb378f service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093031 Aug 24 10:05:12.281103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 155/312] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:12.286619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-4a4d3862-3aad-4458-b4cd-acd63229666e req-3ae625db-9957-41c7-8f43-3798f515a59a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0c601f9d-f26b-4095-a00c-53c422163d31", "tag": "daf8264d-e786-4857-bf7e-3e865d7b48a2"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:12.297484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-4a4d3862-3aad-4458-b4cd-acd63229666e req-3ae625db-9957-41c7-8f43-3798f515a59a service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:12.298275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-4a4d3862-3aad-4458-b4cd-acd63229666e req-3ae625db-9957-41c7-8f43-3798f515a59a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:12.298874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-4a4d3862-3aad-4458-b4cd-acd63229666e req-3ae625db-9957-41c7-8f43-3798f515a59a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:12.311037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-4a4d3862-3aad-4458-b4cd-acd63229666e req-3ae625db-9957-41c7-8f43-3798f515a59a service nova] Creating event network-changed:daf8264d-e786-4857-bf7e-3e865d7b48a2 for instance 0c601f9d-f26b-4095-a00c-53c422163d31 on hv-cinder-85427 Aug 24 10:05:12.321224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-4a4d3862-3aad-4458-b4cd-acd63229666e req-3ae625db-9957-41c7-8f43-3798f515a59a service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038719 Aug 24 10:05:12.322984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 156/313] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:12.328461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:12.336832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:12.354222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:12.354711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:12.355140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:12.356031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:12.356907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:12.358109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:12.723637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:12.739302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:12.742122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:12.746627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b4d759da-7ddc-4683-b4f3-2d5013a28a3c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.422217 Aug 24 10:05:12.749093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 157/314] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:12 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:12.781312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b7543ca5-0c4d-4663-b905-30f7599c8dba tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:12.782117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b7543ca5-0c4d-4663-b905-30f7599c8dba tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:12.941841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b7543ca5-0c4d-4663-b905-30f7599c8dba tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:12.952873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b7543ca5-0c4d-4663-b905-30f7599c8dba tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1246 microversion: 2.1 time: 0.187266 Aug 24 10:05:12.958352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 158/315] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1246 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:12.979633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:12.982756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:13.005665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:13.008608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:13.011056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:13.013482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:13.015877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:13.021205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:13.188556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:13.197728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8846fc9d-38fd-460e-8158-91ed18a34267 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 1.053401 Aug 24 10:05:13.202902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 158/316] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:12 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 1060 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:13.214626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:13.216568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:13.236292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:13.237521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:13.237978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:13.240947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:13.242961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:13.243500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:13.677275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:13.688005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7d52a0d3-0c50-4ce9-8223-23f0ee5f36e3 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.728114 Aug 24 10:05:13.700070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 159/317] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:12 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:13.729071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:13.748707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-08a1a435-805b-43a3-bba0-eb8a8d3852c6 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1313 microversion: 2.1 time: 0.544617 Aug 24 10:05:13.749563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 159/318] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:13 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1313 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:13.778863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:13.779561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:13.791565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:13.792081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:13.792531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:13.793239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:13.793701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:13.794126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:13.979668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:13.992872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:14.008297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:14.012938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:14.013652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:14.014866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:14.020510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:14.021260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:14.173926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:14.188785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a18e53f2-280a-4ab0-afef-56f2c1591808 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1298 microversion: 2.1 time: 0.213686 Aug 24 10:05:14.190670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 160/319] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:13 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1298 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:14.224472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:14.225328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:14.242258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:14.243049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:14.243514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:14.244252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:14.244894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:14.245516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:14.287249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:14.297934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:14.301726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:14.314869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7f0f4ac9-522c-4c3e-805a-d48810a7670e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.541923 Aug 24 10:05:14.315757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 160/320] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:14.360396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-a8c9fe3d-4a40-4565-a6c5-b2e79c15d85f req-0becc694-7248-4609-8296-0573def5b0ec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173", "name": "network-vif-plugged", "status": "completed", "tag": "990aec14-7426-422b-af96-eba82ff28e09"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:14.381136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-a8c9fe3d-4a40-4565-a6c5-b2e79c15d85f req-0becc694-7248-4609-8296-0573def5b0ec service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:14.381618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-a8c9fe3d-4a40-4565-a6c5-b2e79c15d85f req-0becc694-7248-4609-8296-0573def5b0ec service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:14.382043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-a8c9fe3d-4a40-4565-a6c5-b2e79c15d85f req-0becc694-7248-4609-8296-0573def5b0ec service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:14.401420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:14.404353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-a8c9fe3d-4a40-4565-a6c5-b2e79c15d85f req-0becc694-7248-4609-8296-0573def5b0ec service nova] Creating event network-vif-plugged:990aec14-7426-422b-af96-eba82ff28e09 for instance 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 on hv-cinder-85427 Aug 24 10:05:14.423243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-a8c9fe3d-4a40-4565-a6c5-b2e79c15d85f req-0becc694-7248-4609-8296-0573def5b0ec service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066644 Aug 24 10:05:14.424205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 161/321] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:14.466335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=14 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:14.616408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:14.631400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-43199e1a-ba94-4301-9fa4-a9623b3d4bdb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.410572 Aug 24 10:05:14.634612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 161/322] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:14 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:05:14.723479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:14.724225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:14.734882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:14.738086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:14.738777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:14.740418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:14.741433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:14.743602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:14.787929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:14.788820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:14.806007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:14.806828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:14.807394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:14.808354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:14.808828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:14.809543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:15.416470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:15.455723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-12dee66d-3287-480b-b021-b316bc4ae6ae tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.734162 Aug 24 10:05:15.460122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 162/323] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:14 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 741 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:15.478693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:15.481231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:15.490836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:15.491809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:15.493348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:15.494303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:15.495930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:15.497013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:15.497880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:15.508801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-94cc44f2-924d-4778-bed0-5c55f4887d61 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1496 microversion: 2.1 time: 0.729865 Aug 24 10:05:15.509416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 162/324] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:14 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1496 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:15.514219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:15.522763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:15.543474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:15.543973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:15.544435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:15.545134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:15.545586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:15.546004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:16.096609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:16.114983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8cd85a16-3041-4f98-9c07-14d8611aa1b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1306 microversion: 2.1 time: 0.652236 Aug 24 10:05:16.115584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 163/325] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:15 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1306 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:16.137324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:16.138208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:16.151779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:16.153273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:16.153956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:16.154910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:16.155842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:16.156589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:16.541799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:16.550932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:16.559941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:16.561177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7869b89a-bfb8-42e7-af17-5ff877181901 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 1.050652 Aug 24 10:05:16.561736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 163/326] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 1052 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:16.566556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:16.573576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:16.587913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:16.588662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:16.589279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:16.590151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:16.590778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:16.591353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:16.621743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:05:16.789664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9904250a-2623-46a9-aaba-4f4b24b328b2 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "DELETE /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 204 len: 0 microversion: 2.1 time: 0.223303 Aug 24 10:05:16.793022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 164/327] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:05:16 2022] DELETE /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:05:16.814181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:16.814983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:16.836774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:16.839244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:16.839851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:16.844467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:16.845177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:16.845770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:16.960243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:16.979978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-92ea2830-9094-4455-8797-f68dfb728da7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.848382 Aug 24 10:05:16.982881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 164/328] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:16 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 867 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:16.990176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:16.991532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:17.003738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:17.005086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:17.005761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:17.007771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:17.008490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:17.009233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:17.225589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:17.237709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4b2c88db-f6b1-46f3-bedd-ad8fd14138af tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.443073 Aug 24 10:05:17.238281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 165/329] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:16 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:17.246586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:17.249764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:17.261394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:17.268398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:17.271891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:17.273713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:17.278854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:17.279503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:17.489558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:17.505322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-49296336-9502-4841-8b65-f62b875b7ba1 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1502 microversion: 2.1 time: 0.520061 Aug 24 10:05:17.505909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 165/330] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:16 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1502 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:17.578109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:17.588134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-58e53315-2ce2-4e43-94a0-e1615ed28a1e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1304 microversion: 2.1 time: 0.346192 Aug 24 10:05:17.589081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:17.589741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:17.591927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 166/331] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:17 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1304 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:17.611862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:17.624953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:17.625738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:17.626924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:17.627552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:17.628187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:18.006028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:18.007345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:18.025953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:18.026753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:18.029222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:18.030272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:18.032605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:18.033288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:18.090413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:18.099231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:18.120002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:18.121718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c8d5f89d-afd8-4669-96bc-408a5bad6726 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.537074 Aug 24 10:05:18.130635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 166/332] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:18.260882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:18.261871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:18.291419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:18.301403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:18.305090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:18.307073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:18.307695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:18.310700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:18.679750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:18.695364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a95cc417-330a-4582-a161-4137e7f518d8 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1314 microversion: 2.1 time: 0.439381 Aug 24 10:05:18.697308 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 167/333] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:18 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1314 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:18.702254 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:18.713074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:18.720343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:18.722455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:18.724969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:18.727374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:18.733187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:18.734004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:18.758799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:18.781870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-09c5f762-da02-46d6-a6e6-e7197afb4e4d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.779980 Aug 24 10:05:18.796389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 167/334] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:18 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 795 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:05:18.806501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:18.807492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:18.814333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:18.815257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:18.815925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:18.817060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:18.817748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:18.818566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:19.152910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:19.169661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2e94cafa-773d-4fad-a671-55060098e2ba tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1502 microversion: 2.1 time: 0.469642 Aug 24 10:05:19.170858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 168/335] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:18 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1502 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:19.189603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:19.192847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:19.213206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:19.213990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:19.214611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:19.217961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:19.227214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:19.228124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:19.277346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:19.699732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:19.716901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ecdb884c-5516-44b9-b88e-b99990329f59 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.918959 Aug 24 10:05:19.721803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 168/336] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:18 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:19.731588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:19.733456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:19.740555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:19.741052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:19.741492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:19.742195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:19.742651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:19.743590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:19.762776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:19.778287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:19.782500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:19.786675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9e49dded-4cba-4164-a6f5-4c9f1447e097 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.605983 Aug 24 10:05:19.789913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 169/337] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:19.813131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:19.814655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:19.826945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:19.827772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:19.830384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:19.831410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:19.832081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:19.834498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:20.394628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:20.403949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-161394d0-28ca-4311-88b0-8b74931373b6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.595312 Aug 24 10:05:20.408789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 170/338] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:19 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:05:20.413788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:20.420283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:20.436369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:20.438503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:20.439780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:20.441600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:20.443250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:20.445249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:20.835757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:20.853902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e66c3401-e7ce-4840-a77e-6de98d718e50 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1502 microversion: 2.1 time: 0.440153 Aug 24 10:05:20.872140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 171/339] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:20 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1502 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:20.892690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:20.894628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:20.932816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:20.933579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:20.943060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:20.944714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:20.945258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:20.945789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:21.006377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:21.012584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:21.028470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5661c97f-9a09-4d90-a5f6-f6369d8dca61 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1464 microversion: 2.1 time: 1.305094 Aug 24 10:05:21.030227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 169/340] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:19 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1464 bytes in 1307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:21.035832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-53e9db7a-a3c0-4c39-9341-012c756638da req-17dce2be-1194-4de0-97fa-937455e1285b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bdb467be-865b-489c-ab6c-f714a02672e1", "tag": "7b852e03-cec0-4436-b50c-9c652193755e"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:21.082302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-53e9db7a-a3c0-4c39-9341-012c756638da req-17dce2be-1194-4de0-97fa-937455e1285b service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:21.083075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-53e9db7a-a3c0-4c39-9341-012c756638da req-17dce2be-1194-4de0-97fa-937455e1285b service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:21.088566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-53e9db7a-a3c0-4c39-9341-012c756638da req-17dce2be-1194-4de0-97fa-937455e1285b service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:21.103150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-53e9db7a-a3c0-4c39-9341-012c756638da req-17dce2be-1194-4de0-97fa-937455e1285b service nova] Creating event network-changed:7b852e03-cec0-4436-b50c-9c652193755e for instance bdb467be-865b-489c-ab6c-f714a02672e1 on hv-cinder-85427 Aug 24 10:05:21.122325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-53e9db7a-a3c0-4c39-9341-012c756638da req-17dce2be-1194-4de0-97fa-937455e1285b service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.090611 Aug 24 10:05:21.125859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 170/341] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:21.131982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:21.133086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:21.140987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:21.141826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:21.142496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:21.143435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:21.144325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:21.145603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:21.205474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:21.215027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-be32e911-3364-4e41-a4e2-244b64cf85dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.349346 Aug 24 10:05:21.215604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 172/342] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:20 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:21.221475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:21.222220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:21.228456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:21.229649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:21.230278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:21.231565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:21.232231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:21.232841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:21.306738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:05:21.428694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:21.439179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9eacf322-3898-42ed-8f69-595dfd04b945 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "DELETE /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 204 len: 0 microversion: 2.1 time: 0.221903 Aug 24 10:05:21.440196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 173/343] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:05:21 2022] DELETE /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:05:21.448197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:21.449327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:21.467074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:21.467912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:21.468643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:21.469757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:21.470444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:21.471079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:21.892709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:21.923322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:21.932861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:21.934983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:21.939066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-57791dda-2126-4d6b-a48f-609a34e1e771 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.496123 Aug 24 10:05:21.940707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c6d50aa9-d6be-4893-b7d5-decc8817be98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.808935 Aug 24 10:05:21.941471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 174/344] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:21 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:21.953935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 171/345] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:21 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 827 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:21.954831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:21.956741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:21.975248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:21.978293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:21.978987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:21.980301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:21.982901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:21.983576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:22.073129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:22.264341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:22.277404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8bde0319-5b3f-490d-9d09-2cc849fafc45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1470 microversion: 2.1 time: 0.334482 Aug 24 10:05:22.278781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 175/346] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:21 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1470 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:22.286723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:22.288076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:22.298313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:22.299108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:22.306781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:22.308032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:22.308703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:22.309427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:22.404778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-814a8278-e426-4e6b-9ee5-17c1a628756f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Action: 'create', calling method: .version_select at 0x7fa33e3d4c10>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1474637294", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFC14K2TbiKtipAHyyUO7u7HHClxYdqy5EumCg5t6rdNTUEh10giKYPKyyzLAbnxG5XvJAyhJ9G1A8eNefzhDog95t03xkTciJBy1U8wJrnZoaiZYBHCV/PCUy83qZXT/A=="}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:22.424267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '731cf6df5bda42148e1420e365306445', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12553f79dbd4d75a291d3a07a8dbbba', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:22.436661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-814a8278-e426-4e6b-9ee5-17c1a628756f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Getting quotas for project dc33f347b062491aaa04b7dc18553cf4. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:05:22.439298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d6059b41-af14-43e8-acd7-c3fd1b09a3c8 tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 200 len: 1460 microversion: 2.1 time: 0.158042 Aug 24 10:05:22.441948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-814a8278-e426-4e6b-9ee5-17c1a628756f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Getting quotas for user 3ccfd2f7e35646b0a60b28b6807cee4b and project dc33f347b062491aaa04b7dc18553cf4. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:05:22.453319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 176/347] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:22 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 1460 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:22.458300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:22.458914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:22.476762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-814a8278-e426-4e6b-9ee5-17c1a628756f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.521474 Aug 24 10:05:22.479692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 172/348] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:05:21 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:22.481635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:22.482315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:22.482996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:22.483922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:22.484621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:22.485212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:22.640795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-04532a54-cd94-4ccd-800a-daae5525444c req-d1d826e4-5bdb-440b-a8cd-0c0d92ce8926 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "47745245-e791-4bdd-b31d-6f745be13913", "name": "network-vif-deleted", "tag": "860d5021-b87f-44d1-9b8a-9c8005fd4363"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:22.663035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-04532a54-cd94-4ccd-800a-daae5525444c req-d1d826e4-5bdb-440b-a8cd-0c0d92ce8926 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:22.663822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-04532a54-cd94-4ccd-800a-daae5525444c req-d1d826e4-5bdb-440b-a8cd-0c0d92ce8926 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:22.664641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-04532a54-cd94-4ccd-800a-daae5525444c req-d1d826e4-5bdb-440b-a8cd-0c0d92ce8926 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:22.720505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-04532a54-cd94-4ccd-800a-daae5525444c req-d1d826e4-5bdb-440b-a8cd-0c0d92ce8926 service nova] Creating event network-vif-deleted:860d5021-b87f-44d1-9b8a-9c8005fd4363 for instance 47745245-e791-4bdd-b31d-6f745be13913 on hv-cinder-85427 Aug 24 10:05:22.749007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-04532a54-cd94-4ccd-800a-daae5525444c req-d1d826e4-5bdb-440b-a8cd-0c0d92ce8926 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.111050 Aug 24 10:05:22.755938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 173/349] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:22.948220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=22 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:22.968404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:22.969644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:22.988366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:22.988915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:22.993920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:22.996236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:22.996734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:22.997232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:22.997487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:23.008849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d8bbdbf5-0a0c-4f0c-8928-2b5bb3b7d92e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.551057 Aug 24 10:05:23.010025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 177/350] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:22 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:23.019848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:23.022997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:23.042993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:23.048803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:23.049256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:23.049993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:23.050455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:23.050881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:23.923303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:23.933498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:23.936203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6c35c93c-9fa2-4af2-9850-843e6ef7c024 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.977355 Aug 24 10:05:23.939816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 174/351] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:22 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 983 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:23.947244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:23.953825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:23.969587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:23.973335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:23.974104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:23.975939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:23.979031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:23.979758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:23.990521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:24.001602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:24.003246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2c7a3e30-45cc-449a-866e-a55996c2e9e4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.991963 Aug 24 10:05:24.017466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 178/352] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:23 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 1006 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:24.030334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:24.031063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] [instance: 47745245-e791-4bdd-b31d-6f745be13913] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:24.059455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:24.067877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:24.067877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:24.067877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:24.067877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:24.068859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:24.100448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] HTTP exception thrown: Instance 47745245-e791-4bdd-b31d-6f745be13913 could not be found. Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Request method failure captured: Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 HTTP/1.1 Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfeuW7pI2kYhP44XP-MRTEr2eD_1700H-cXU5l5eTLTqIds9t2Vild7swpbEX9UaBuyYBTb5bUQWIpHUTB_4D8rbSKC9K6B7UBBhrL6E7uzgysDpSq29VGhp0voj-JrQPskFR0xvvYCDujfAkV-Cuds76ITbaRkNre1k_lJbj_1kAKs Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: f12553f79dbd4d75a291d3a07a8dbbba Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesActionsTest-1093348380 Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:05:24.102783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f12553f79dbd4d75a291d3a07a8dbbba"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f12553f79dbd4d75a291d3a07a8dbbba"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f12553f79dbd4d75a291d3a07a8dbbba"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f12553f79dbd4d75a291d3a07a8dbbba"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesActionsTest-1093348380 Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: f12553f79dbd4d75a291d3a07a8dbbba Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesActionsTest-1093348380 Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesActionsTest-1093348380-project Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 731cf6df5bda42148e1420e365306445 Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesActionsTest-1093348380-project Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 47745245-e791-4bdd-b31d-6f745be13913 could not be found. Aug 24 10:05:24.104203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '47745245-e791-4bdd-b31d-6f745be13913'} Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 47745245-e791-4bdd-b31d-6f745be13913 could not be found. Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:05:24.104992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 47745245-e791-4bdd-b31d-6f745be13913 could not be found. Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 47745245-e791-4bdd-b31d-6f745be13913 could not be found. Aug 24 10:05:24.106488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:05:24.109778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] Returning 404 to user: Instance 47745245-e791-4bdd-b31d-6f745be13913 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:05:24.109778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e2768c44-f5fb-4c51-828d-b832ecea380a tempest-VolumesActionsTest-1093348380 tempest-VolumesActionsTest-1093348380-project] 10.222.0.86 "GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913" status: 404 len: 111 microversion: 2.1 time: 0.086687 Aug 24 10:05:24.110967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 179/353] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:24 2022] GET /compute/v2.1/servers/47745245-e791-4bdd-b31d-6f745be13913 => generated 111 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:24.116507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:24.117533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:24.134939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:24.136612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:24.138563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:24.140195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:24.143192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:24.143714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:24.426924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:24.435592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-74fed669-95a0-484a-91c3-ece405278a3d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.323306 Aug 24 10:05:24.442170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 180/354] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:24 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:24.582628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:24.601594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-88ed89aa-6315-421b-af1a-28465be4e27d tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1470 microversion: 2.1 time: 0.660459 Aug 24 10:05:24.604855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 175/355] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:23 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1470 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:24.961627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:24.962249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:24.968946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:24.969447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:24.969878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:24.976552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:24.980912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:24.981685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:25.031647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:25.032568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:25.049175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:25.049688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:25.050097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:25.050800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:25.051251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:25.051676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:25.430179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:25.463797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-33e2c22f-0600-4ec2-91e2-eb807d50f053 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.506654 Aug 24 10:05:25.489560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 181/356] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:24 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:25.512191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:25.512999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:25.541704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:25.542436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:25.543122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:25.544064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:25.545473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:25.546389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:25.823376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:25.837752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:25.844342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:25.853213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a1bedc64-3f77-4896-901d-d391fa03efa9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.825354 Aug 24 10:05:25.858074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 176/357] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:25 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:25.868337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:25.871801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:25.883800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:25.885650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cccd9cc6-9522-4308-93ca-0e2d6c9932be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.391546 Aug 24 10:05:25.886598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 182/358] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:25 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:25.901816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:25.904650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:25.909043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:25.911156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:25.911748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:25.912565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:26.141501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-5b209112-0b64-47fd-b6fb-9aa817301514 req-653bac88-051f-43fb-9b50-676638f70cfb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173", "name": "network-vif-deleted", "tag": "990aec14-7426-422b-af96-eba82ff28e09"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:26.192798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5b209112-0b64-47fd-b6fb-9aa817301514 req-653bac88-051f-43fb-9b50-676638f70cfb service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:26.193494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5b209112-0b64-47fd-b6fb-9aa817301514 req-653bac88-051f-43fb-9b50-676638f70cfb service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:26.194051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5b209112-0b64-47fd-b6fb-9aa817301514 req-653bac88-051f-43fb-9b50-676638f70cfb service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:26.214559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:26.223180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8618a993-744c-4fa3-a999-043515165a45 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1428 microversion: 2.1 time: 0.363611 Aug 24 10:05:26.227817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-5b209112-0b64-47fd-b6fb-9aa817301514 req-653bac88-051f-43fb-9b50-676638f70cfb service nova] Creating event network-vif-deleted:990aec14-7426-422b-af96-eba82ff28e09 for instance 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 on hv-cinder-85427 Aug 24 10:05:26.228757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 177/359] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:25 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1428 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:26.239222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-5b209112-0b64-47fd-b6fb-9aa817301514 req-653bac88-051f-43fb-9b50-676638f70cfb service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.104699 Aug 24 10:05:26.239829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 183/360] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:26.486947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:26.488116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:26.495764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:26.496537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:26.497238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:26.501647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:26.502446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:26.503076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:26.842590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:26.851060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f77b0f4d-3924-47a7-97a4-56001b5fb5f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.367993 Aug 24 10:05:26.851633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 178/361] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:26 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:26.888913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:26.889954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:26.907532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:26.908178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:26.908862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:26.909434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:26.912267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:26.914909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:26.918748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:26.919530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:26.922936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:26.923258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:26.923702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:26.924439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:26.924900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:26.930724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:27.357052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:27.363537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:27.375728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-213930c3-5512-4086-9202-af9be8549004 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.473109 Aug 24 10:05:27.376189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:27.379107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 179/362] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:26 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:27.382024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:27.384521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d2d9e2e2-acd5-49ed-8d17-095991b718c3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.504768 Aug 24 10:05:27.387086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 184/363] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:26 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:27.400325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:27.401213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:27.410121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:27.411452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:27.412063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:27.412955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:27.414012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:27.414633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:27.547346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:27.558201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1aef8a5b-bdef-42f1-b779-1dd33a1717c2 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1428 microversion: 2.1 time: 0.177732 Aug 24 10:05:27.572250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 180/364] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:27 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1428 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:27.876084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:27.876434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:27.887345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:27.892685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:27.896133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:27.901166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:27.902650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:27.903270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:28.226157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:28.230022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:28.241393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6d02f4cf-6c0f-40f8-a8df-303d4983489e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.370119 Aug 24 10:05:28.256799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 185/365] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:27 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:28.404498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:28.413102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:28.414184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:28.414882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:28.436567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:28.437810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:28.439589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:28.441116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:28.442354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:28.443430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:28.444705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:28.445482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:28.447056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:28.448082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:28.449741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:28.452665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:28.762437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:28.773432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:28.779099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:28.781233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aff1b132-6f1e-4d54-9b0c-82e80eb1a68e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.372507 Aug 24 10:05:28.782014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 186/366] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:28 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:28.787623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:28.801889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:28.825601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:28.831005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:28.833238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:28.834544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:28.837299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:28.837996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:28.907948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:28.909164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:28.909890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:28.910506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lazy-loading 'fault' on Instance uuid 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:05:28.998790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:29.009582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-38da6bac-ef19-4e1e-8e8d-5fc52a32ed5c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.609299 Aug 24 10:05:29.013307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 181/367] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:28 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:29.054048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2063027ade874ab19c7df1655ebd6b85', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b3d697163a8d4ee197f1b6b2856f08ae', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:29.064259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1f572c47-030f-4ce2-95f5-1c6b1c61dc76 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 200 len: 1304 microversion: 2.1 time: 0.280792 Aug 24 10:05:29.068899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 187/368] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:28 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 1304 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:29.087262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d6fc8120-0176-4c0d-bbc0-c977600800c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:29.087997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d6fc8120-0176-4c0d-bbc0-c977600800c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:29.098734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6fc8120-0176-4c0d-bbc0-c977600800c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:29.100226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6fc8120-0176-4c0d-bbc0-c977600800c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:29.100905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6fc8120-0176-4c0d-bbc0-c977600800c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:29.101889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6fc8120-0176-4c0d-bbc0-c977600800c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:29.104300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6fc8120-0176-4c0d-bbc0-c977600800c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:29.104932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6fc8120-0176-4c0d-bbc0-c977600800c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:29.171467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d6fc8120-0176-4c0d-bbc0-c977600800c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/ips" status: 200 len: 147 microversion: 2.1 time: 0.089977 Aug 24 10:05:29.186960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 182/369] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:05:29 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/ips => generated 147 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:29.267647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:29.268954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:29.284805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:29.285827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:29.288355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:29.291218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:29.298114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:29.298472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:29.624697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:29.648176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d3ca2ae9-fe2c-4e6e-b6a9-cbc6c289f81c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.386726 Aug 24 10:05:29.651076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 188/370] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:29 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:29.805836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:29.806461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:29.830886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:29.831784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:29.832987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:29.833967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:29.835201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:29.837544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:30.038947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:30.047064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:30.058604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:30.059704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:30.061331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:30.064306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:30.065013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:30.071670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:30.191237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:30.206513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:30.208387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:30.209626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2b5cba0c-7255-4550-8df1-b3bd37f9057e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.409328 Aug 24 10:05:30.223290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 183/371] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:30.237207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:30.247834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] [instance: 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:30.279844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:30.280382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:30.286718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:30.290960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:30.291457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:30.291904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:30.377418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] HTTP exception thrown: Instance 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 could not be found. Aug 24 10:05:30.378645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Request method failure captured: Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 HTTP/1.1 Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffAWxsiQnge0-9Vu9zkixcOrn_UDs-lPntQE2-aODGQRh-45Vdw84EDSsoAMORQF5S0wqcA0k4ch3eUXM8xK0EKHc4asEUZvJahMv7ZDbKA2rqRK06Hb4tnfTp1l_JNeNvPCfOHwBdtIVvwTZOxbq0qd6IsaAT7sRQ2A2OtiGxs69o Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: b3d697163a8d4ee197f1b6b2856f08ae Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesNegativeTest-436211697 Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:05:30.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/b3d697163a8d4ee197f1b6b2856f08ae"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/b3d697163a8d4ee197f1b6b2856f08ae"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_b3d697163a8d4ee197f1b6b2856f08ae"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/b3d697163a8d4ee197f1b6b2856f08ae"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesNegativeTest-436211697 Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: b3d697163a8d4ee197f1b6b2856f08ae Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesNegativeTest-436211697 Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesNegativeTest-436211697-project Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 2063027ade874ab19c7df1655ebd6b85 Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesNegativeTest-436211697-project Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 could not be found. Aug 24 10:05:30.399457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173'} Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 could not be found. Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:05:30.400603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 could not be found. Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 could not be found. Aug 24 10:05:30.402149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:05:30.412980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d67ede91-8636-4bc9-9403-6db34a744a64 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.375561 Aug 24 10:05:30.415122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] Returning 404 to user: Instance 8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:05:30.416035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-445d5aa2-d50a-4e42-9227-8433415b6657 tempest-VolumesNegativeTest-436211697 tempest-VolumesNegativeTest-436211697-project] 10.222.0.86 "GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173" status: 404 len: 111 microversion: 2.1 time: 0.191211 Aug 24 10:05:30.424374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 184/372] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:30 2022] GET /compute/v2.1/servers/8e5b7d1b-1c63-4cf4-ba9a-b0c1108c6173 => generated 111 bytes in 197 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:30.435265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 189/373] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:30 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:30.498560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-dbcecca2-815c-415e-8094-0dde1b3ce4d5 req-5d323920-fd6f-4060-b94c-e10c363f9948 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0c601f9d-f26b-4095-a00c-53c422163d31", "name": "network-vif-plugged", "status": "completed", "tag": "daf8264d-e786-4857-bf7e-3e865d7b48a2"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:30.509341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-dbcecca2-815c-415e-8094-0dde1b3ce4d5 req-5d323920-fd6f-4060-b94c-e10c363f9948 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:30.510436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-dbcecca2-815c-415e-8094-0dde1b3ce4d5 req-5d323920-fd6f-4060-b94c-e10c363f9948 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:30.510886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-dbcecca2-815c-415e-8094-0dde1b3ce4d5 req-5d323920-fd6f-4060-b94c-e10c363f9948 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:30.524949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-dbcecca2-815c-415e-8094-0dde1b3ce4d5 req-5d323920-fd6f-4060-b94c-e10c363f9948 service nova] Creating event network-vif-plugged:daf8264d-e786-4857-bf7e-3e865d7b48a2 for instance 0c601f9d-f26b-4095-a00c-53c422163d31 on hv-cinder-85427 Aug 24 10:05:30.531458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-dbcecca2-815c-415e-8094-0dde1b3ce4d5 req-5d323920-fd6f-4060-b94c-e10c363f9948 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036401 Aug 24 10:05:30.532506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 185/374] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:30.683450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:30.684496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:30.695015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:30.696444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:30.697181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:30.698987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:30.700091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:30.700708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:30.968283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:30.995093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3fb99868-3fec-49a6-b987-a85c2a6281c5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.317532 Aug 24 10:05:31.009553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 190/375] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:30 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:31.252100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:31.252826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:31.254845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:31.291382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:31.291692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:31.292931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:31.293880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:31.294534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:31.295674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:31.432996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:31.433984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:31.451021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:31.454146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:31.462845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:31.464483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:31.465157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:31.465841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:31.736946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:31.748346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:31.748774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:31.749993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d25ea2e0-846c-42f5-a784-3ca53de1865b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.502736 Aug 24 10:05:31.771562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 186/376] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:31 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:31.854259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:31.868389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dab4c99c-7a32-4116-8111-e5eae88536e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.440802 Aug 24 10:05:31.869912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 191/377] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:31 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:32.020840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:32.021787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:32.033800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:32.034622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:32.035481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:32.036580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:32.038020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:32.038681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:32.452411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:32.655458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:32.663867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-27c7b293-a045-4c11-85d0-284827c8a954 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1260 microversion: 2.1 time: 0.649496 Aug 24 10:05:32.665007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 187/378] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:32 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1260 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:32.706189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-577c9538-1d00-4819-8a8b-2407289774c5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "678c737f-2b39-4a8b-b65e-8026cc57f466"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:32.708868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-577c9538-1d00-4819-8a8b-2407289774c5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:32.728380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-577c9538-1d00-4819-8a8b-2407289774c5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:32.730347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-577c9538-1d00-4819-8a8b-2407289774c5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:32.732805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-577c9538-1d00-4819-8a8b-2407289774c5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:32.734012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-577c9538-1d00-4819-8a8b-2407289774c5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:32.737101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-577c9538-1d00-4819-8a8b-2407289774c5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:32.737949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-577c9538-1d00-4819-8a8b-2407289774c5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:32.778320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:32.779244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:32.801757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:32.802318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:32.802687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:32.803401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:32.803858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:32.804310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:33.082606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:33.106945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:33.110592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:33.120503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2e697ebc-c0d7-4b58-a87f-87ab9a5f312b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.341264 Aug 24 10:05:33.132155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 188/379] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:32 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:33.132155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:33.132155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:33.146998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:33.147509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:33.147952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:33.156799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:33.157482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:33.161231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:33.378294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-577c9538-1d00-4819-8a8b-2407289774c5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.676123 Aug 24 10:05:33.379357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 192/380] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:05:32 2022] POST /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 194 bytes in 678 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:33.518442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:33.527526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b89d8b6c-b5d7-4e4d-94e3-4adfc683fdb2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.404932 Aug 24 10:05:33.529677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 189/381] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:33 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:33.689711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:33.690511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:33.702529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:33.705252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:33.706240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:33.706986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:33.710084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:33.710730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:34.037107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:34.047976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b6fa5bfe-c3c2-429f-a148-1467291ec86a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1443 microversion: 2.1 time: 0.365406 Aug 24 10:05:34.056632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 193/382] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:33 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1443 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:34.072649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:34.073691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:34.089999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:34.092071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:34.093029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:34.097381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:34.108432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:34.110879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:34.148720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:34.150778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:34.163348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:34.165561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:34.166365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:34.167711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:34.169186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:34.171307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:34.172325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:34.514251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:34.525337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a1c1f9f0-2fc5-48fa-9755-089a8868f437 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1443 microversion: 2.1 time: 0.455651 Aug 24 10:05:34.526780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 190/383] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:34 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1443 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:34.562479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:34.562782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:34.581258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:34.581767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:34.582210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:34.582928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:34.583384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:34.583809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:34.727789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:34.748457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:34.752782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:34.755904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-560d962c-10a7-4a5d-9cab-971bd4739481 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.612622 Aug 24 10:05:34.762650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 194/384] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:34 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:34.771434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-7a1539b9-548a-4985-87aa-0c29728cbf8e req-dbd0aa3f-4a27-4a3d-83ba-43b95b2b4288 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bdb467be-865b-489c-ab6c-f714a02672e1", "name": "network-vif-plugged", "status": "completed", "tag": "7b852e03-cec0-4436-b50c-9c652193755e"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:34.804219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-7a1539b9-548a-4985-87aa-0c29728cbf8e req-dbd0aa3f-4a27-4a3d-83ba-43b95b2b4288 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:34.805095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-7a1539b9-548a-4985-87aa-0c29728cbf8e req-dbd0aa3f-4a27-4a3d-83ba-43b95b2b4288 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:34.806995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-7a1539b9-548a-4985-87aa-0c29728cbf8e req-dbd0aa3f-4a27-4a3d-83ba-43b95b2b4288 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:34.824544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:34.839573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5bffd74a-8269-41fd-b84b-a9d0f27b28a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.282125 Aug 24 10:05:34.840414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 191/385] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:34 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:34.847804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-7a1539b9-548a-4985-87aa-0c29728cbf8e req-dbd0aa3f-4a27-4a3d-83ba-43b95b2b4288 service nova] Creating event network-vif-plugged:7b852e03-cec0-4436-b50c-9c652193755e for instance bdb467be-865b-489c-ab6c-f714a02672e1 on hv-cinder-85427 Aug 24 10:05:34.861158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-7a1539b9-548a-4985-87aa-0c29728cbf8e req-dbd0aa3f-4a27-4a3d-83ba-43b95b2b4288 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096986 Aug 24 10:05:34.862362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 195/386] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:35.795281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:35.796831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:35.811986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:35.812518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:35.812963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:35.813686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:35.814143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:35.814569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:35.864126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:35.865407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:35.883742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:35.884726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:35.885325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:35.886228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:35.886831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:35.887536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:36.199711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:36.213277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-81f44213-3e53-4845-ba98-3200ab07e2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.354136 Aug 24 10:05:36.219105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 196/387] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:35 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:05:36.517374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:36.526822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:36.535200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:36.536849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e16ca386-c043-4ad3-bb7e-66ad7f7148a9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.749663 Aug 24 10:05:36.538521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 192/388] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:35 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:37.234196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:37.236147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:37.244315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:37.245152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:37.245828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:37.246747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:37.247405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:37.248034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:37.493228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:37.501841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6c8ac540-f4c7-46a5-9565-7b4fee490d85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1372 microversion: 2.1 time: 0.271554 Aug 24 10:05:37.502988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 197/389] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:37 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1372 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:37.562391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:37.563008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:37.575268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:37.576011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:37.576470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:37.577320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:37.577928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:37.578559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:38.085559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:38.096697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:38.104617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:38.107046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0db9f2db-f085-4e90-8713-88e658352d42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.548893 Aug 24 10:05:38.107632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 193/390] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:38.333004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-8b623548-a6e0-492a-8ead-898f2098c1c8 req-4643cb53-7d30-4dcf-8897-bb3698f538ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0c601f9d-f26b-4095-a00c-53c422163d31", "tag": "daf8264d-e786-4857-bf7e-3e865d7b48a2"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:38.353587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-8b623548-a6e0-492a-8ead-898f2098c1c8 req-4643cb53-7d30-4dcf-8897-bb3698f538ad service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:38.354214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-8b623548-a6e0-492a-8ead-898f2098c1c8 req-4643cb53-7d30-4dcf-8897-bb3698f538ad service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:38.354745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-8b623548-a6e0-492a-8ead-898f2098c1c8 req-4643cb53-7d30-4dcf-8897-bb3698f538ad service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:38.387992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-8b623548-a6e0-492a-8ead-898f2098c1c8 req-4643cb53-7d30-4dcf-8897-bb3698f538ad service nova] Creating event network-changed:daf8264d-e786-4857-bf7e-3e865d7b48a2 for instance 0c601f9d-f26b-4095-a00c-53c422163d31 on hv-cinder-85427 Aug 24 10:05:38.397080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-8b623548-a6e0-492a-8ead-898f2098c1c8 req-4643cb53-7d30-4dcf-8897-bb3698f538ad service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068010 Aug 24 10:05:38.398598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 198/391] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:38.526817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:38.527637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:38.542044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:38.544874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:38.548021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:38.549117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:38.553107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:38.553902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:38.974111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:38.987560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8e36737c-53b2-4878-8b62-091a79199f63 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1565 microversion: 2.1 time: 0.464466 Aug 24 10:05:38.991361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 194/392] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:38 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1565 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:39.015389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-36887579-00a1-43b0-8725-10fbfb3a7323 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:39.016400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-36887579-00a1-43b0-8725-10fbfb3a7323 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:39.033509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36887579-00a1-43b0-8725-10fbfb3a7323 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:39.034410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36887579-00a1-43b0-8725-10fbfb3a7323 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:39.035091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36887579-00a1-43b0-8725-10fbfb3a7323 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:39.036103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36887579-00a1-43b0-8725-10fbfb3a7323 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:39.036795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36887579-00a1-43b0-8725-10fbfb3a7323 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:39.037461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36887579-00a1-43b0-8725-10fbfb3a7323 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:39.129661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:39.130801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:39.169979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:39.170495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:39.170936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:39.171655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:39.172128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:39.181281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:39.240857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-36887579-00a1-43b0-8725-10fbfb3a7323 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.230655 Aug 24 10:05:39.248897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 199/393] 10.222.0.86 () {58 vars in 1275 bytes} [Wed Aug 24 10:05:39 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-interface => generated 284 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:05:39.493324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:39.502896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:39.506185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:39.511189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f6207f0b-f258-4a07-884c-c9ec3f653e3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.385345 Aug 24 10:05:39.514872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 195/394] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:39 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:40.567188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:40.574001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:40.592310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:40.608933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:40.609915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:40.611345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:40.614125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:40.624464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:41.103119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:41.125434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-3f1ff335-740d-4739-89cb-196e071e7e70 req-150b72bb-99de-4252-b2d8-9cceb32e591a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bdb467be-865b-489c-ab6c-f714a02672e1", "tag": "7b852e03-cec0-4436-b50c-9c652193755e"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:41.130455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:41.142190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:41.147596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a5f42e80-28eb-4f43-a839-cc99143ce629 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.608238 Aug 24 10:05:41.149910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 200/395] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:41.154147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-3f1ff335-740d-4739-89cb-196e071e7e70 req-150b72bb-99de-4252-b2d8-9cceb32e591a service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:41.155150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-3f1ff335-740d-4739-89cb-196e071e7e70 req-150b72bb-99de-4252-b2d8-9cceb32e591a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:41.157108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-3f1ff335-740d-4739-89cb-196e071e7e70 req-150b72bb-99de-4252-b2d8-9cceb32e591a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:41.179115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-3f1ff335-740d-4739-89cb-196e071e7e70 req-150b72bb-99de-4252-b2d8-9cceb32e591a service nova] Creating event network-changed:7b852e03-cec0-4436-b50c-9c652193755e for instance bdb467be-865b-489c-ab6c-f714a02672e1 on hv-cinder-85427 Aug 24 10:05:41.184975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-3f1ff335-740d-4739-89cb-196e071e7e70 req-150b72bb-99de-4252-b2d8-9cceb32e591a service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066577 Aug 24 10:05:41.186902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 196/396] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:05:41.642197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b6e852c7-828f-410b-bd66-c82ddd9d6b1a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:41.651170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6e852c7-828f-410b-bd66-c82ddd9d6b1a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:41.652253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6e852c7-828f-410b-bd66-c82ddd9d6b1a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:41.653262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6e852c7-828f-410b-bd66-c82ddd9d6b1a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:41.762764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b6e852c7-828f-410b-bd66-c82ddd9d6b1a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Going to try to stop instance {{(pid=120654) force_stop /opt/stack/nova/nova/compute/api.py:2749}} Aug 24 10:05:41.808392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:41.854332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b6e852c7-828f-410b-bd66-c82ddd9d6b1a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/action" status: 202 len: 0 microversion: 2.1 time: 0.212037 Aug 24 10:05:41.858848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 201/397] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:05:41 2022] POST /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/action => generated 0 bytes in 223 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:05:41.865600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:41.866273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:41.892976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:41.893559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:41.894019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:41.898899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:41.899424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:41.899886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:42.173795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:42.174789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:42.190074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:42.190569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:42.196330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:42.197402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:42.197977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:42.198456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:42.437171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:42.453785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b5fca296-9f33-413b-9dbe-005d5c7f6744 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1724 microversion: 2.1 time: 0.585444 Aug 24 10:05:42.458803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 197/398] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:41 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1724 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:42.570739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:42.596954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:42.599551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:42.602012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-085bc2c6-752e-4361-8998-3ba58f430444 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1741 microversion: 2.20 time: 0.433995 Aug 24 10:05:42.604037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 202/399] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:42 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1741 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:43.493383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:43.494207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:43.530592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:43.531912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:43.533293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:43.534244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:43.534865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:43.535455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:43.631708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:43.633769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:43.665214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:43.666259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:43.667375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:43.668445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:43.669918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:43.676401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:43.893764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:43.912247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3ca40f85-512c-4bf7-b3d9-a9ff6484aa46 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1724 microversion: 2.1 time: 0.431173 Aug 24 10:05:43.924085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 198/400] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:43 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1724 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:43.999781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:44.015857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:44.024943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:44.026242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c77bf0ae-355d-47f7-902c-b273e21e2f20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1723 microversion: 2.20 time: 0.400038 Aug 24 10:05:44.047741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 203/401] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1723 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:44.937144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:44.938317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:44.954200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:44.955349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:44.956922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:44.959502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:44.963619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:44.966285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:45.054254 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:45.054833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:45.066467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:45.066955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:45.067387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:45.068127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:45.070297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:45.070943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:45.261285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:45.273521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cd14b29e-c0d6-4a2c-a1e3-27f1e19d56bd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1724 microversion: 2.1 time: 0.339437 Aug 24 10:05:45.288761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 199/402] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:44 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1724 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:45.394328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:45.406384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:45.408071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:45.420021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8ed6025d-1470-4202-892f-9bc52d10bc2f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1723 microversion: 2.20 time: 0.370029 Aug 24 10:05:45.421298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 204/403] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:45 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1723 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:46.296367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:46.297956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:46.316814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:46.317744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:46.318612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:46.319532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:46.320150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:46.321009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:46.447960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:46.448689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:46.468856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:46.470143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:46.470947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:46.471872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:46.472524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:46.473116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:46.837586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:46.846546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ae766d50-7e50-4838-8b6f-9ac97c43251c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1724 microversion: 2.1 time: 0.554971 Aug 24 10:05:46.851854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 200/404] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:46 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1724 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:46.885932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:46.901606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:46.903303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:46.905070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-02a7b831-7096-4ae4-b6d0-c154e7e5c5ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1723 microversion: 2.20 time: 0.461087 Aug 24 10:05:46.921654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 205/405] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:46 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1723 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:47.884492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:47.884492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:47.886076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-f6ab370e-377b-4595-9568-9cc2b9e14dab req-2056c000-2fd7-43dd-a616-3b0ce90d98d7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1", "name": "network-vif-unplugged", "status": "completed", "tag": "1b9069bd-2630-4b65-9e91-d7695e3b6a8c"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:47.906838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:47.908711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:47.913682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:47.924870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:47.924870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:47.924870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:47.927445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f6ab370e-377b-4595-9568-9cc2b9e14dab req-2056c000-2fd7-43dd-a616-3b0ce90d98d7 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:47.929602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f6ab370e-377b-4595-9568-9cc2b9e14dab req-2056c000-2fd7-43dd-a616-3b0ce90d98d7 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:47.929602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f6ab370e-377b-4595-9568-9cc2b9e14dab req-2056c000-2fd7-43dd-a616-3b0ce90d98d7 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:47.970927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.compute.server_external_events [req-f6ab370e-377b-4595-9568-9cc2b9e14dab req-2056c000-2fd7-43dd-a616-3b0ce90d98d7 service nova] Unable to find a host for instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1. Dropping event network-vif-unplugged {{(pid=120654) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Aug 24 10:05:47.975986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-f6ab370e-377b-4595-9568-9cc2b9e14dab req-2056c000-2fd7-43dd-a616-3b0ce90d98d7 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.093796 Aug 24 10:05:47.979493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 206/406] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:05:47 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 99 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Aug 24 10:05:47.992563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:47.993781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:48.005072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:48.005568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:48.006002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:48.006743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:48.025406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:48.025406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:48.185157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:48.198689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f948810e-064f-4eec-8f65-252f64ead9a7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1724 microversion: 2.1 time: 0.323537 Aug 24 10:05:48.204489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 201/407] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:47 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1724 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:48.227464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:48.242142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:48.244266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:48.245114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3d458d63-6f9a-4450-be11-6ffe4462d234 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1666 microversion: 2.20 time: 0.263795 Aug 24 10:05:48.245683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 207/408] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1666 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:48.269671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d0078b20-3c1f-4f49-b46c-fe27dc992fde tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c199d515-fa41-43ae-9e8f-cd84304ee49d"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:48.273241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d0078b20-3c1f-4f49-b46c-fe27dc992fde tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:48.280650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0078b20-3c1f-4f49-b46c-fe27dc992fde tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:48.281449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0078b20-3c1f-4f49-b46c-fe27dc992fde tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:48.282093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0078b20-3c1f-4f49-b46c-fe27dc992fde tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:48.283016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0078b20-3c1f-4f49-b46c-fe27dc992fde tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:48.283663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0078b20-3c1f-4f49-b46c-fe27dc992fde tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:48.284308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0078b20-3c1f-4f49-b46c-fe27dc992fde tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:48.703540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d0078b20-3c1f-4f49-b46c-fe27dc992fde tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "POST /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.438060 Aug 24 10:05:48.710908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 202/409] 10.222.0.86 () {62 vars in 1364 bytes} [Wed Aug 24 10:05:48 2022] POST /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 188 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:05:48.883263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:48.884227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:48.891163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:48.892469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:48.893155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:48.894111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:48.894780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:48.895401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:48.953343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120654) unshelve /opt/stack/nova/nova/compute/api.py:4578}} Aug 24 10:05:49.070916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a429b703-f3c4-48fc-94e7-2290c5aa200e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "POST /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/action" status: 202 len: 0 microversion: 2.20 time: 0.187236 Aug 24 10:05:49.072761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 208/410] 10.222.0.86 () {62 vars in 1319 bytes} [Wed Aug 24 10:05:48 2022] POST /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/action => generated 0 bytes in 194 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:05:49.080367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:49.081312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:49.089406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:49.090193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:49.090907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:49.093851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:49.094803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:49.095790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:49.226160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:49.227116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:49.241626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:49.242502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:49.244403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:49.246412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:49.249320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:49.250563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:49.344384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:49.363509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:49.367665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:49.385191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-587a1e2a-20a2-4cb8-bacc-340b50ca24b9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1752 microversion: 2.20 time: 0.308707 Aug 24 10:05:49.387004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 203/411] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1752 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:49.818295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:49.837234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a60ae841-1c9a-458f-b9ab-ff0a1e4a8c20 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1724 microversion: 2.1 time: 0.613168 Aug 24 10:05:49.838718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 209/412] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:49 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1724 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:50.411544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:50.412284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:50.423749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:50.424393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:50.424903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:50.425680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:50.426212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:50.426707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:50.623071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:50.632236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:50.642338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:50.653350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4f401826-766b-4632-90c8-8009d6756a38 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1752 microversion: 2.20 time: 0.246291 Aug 24 10:05:50.654407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 204/413] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:50 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1752 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:50.885811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:50.885811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:50.900330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:50.900974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:50.901516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:50.906735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:50.907702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:50.912400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:51.128249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:51.138158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fe143779-5d93-4fa1-a7d9-4f5e42a915cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1724 microversion: 2.1 time: 0.278590 Aug 24 10:05:51.152070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 210/414] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:50 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1724 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:51.681734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:51.682991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:51.689470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:51.689928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:51.690467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:51.691366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:51.691818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:51.692335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:51.697665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:51.899793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:51.909475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:51.911233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:51.912688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3ec140ca-778e-41d6-88e4-996172e62402 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1752 microversion: 2.20 time: 0.235584 Aug 24 10:05:51.934587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 205/415] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1752 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:52.160617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:52.161235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:52.164204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=83,nova_api:UPDATE=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:52.171371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:52.171860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:52.172319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:52.173523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:52.173986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:52.174412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:52.368822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:52.379490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1d64778b-df0a-4108-869e-951d72f6cad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1724 microversion: 2.1 time: 0.222622 Aug 24 10:05:52.403650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 211/416] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:52 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1724 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:52.939840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:52.941494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:52.950212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:52.951359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:52.952075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:52.953482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:52.954194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:52.955117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:53.263687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:53.281584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:53.283575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:53.285382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-053906e5-1754-4235-a68d-fa18f28d8146 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1752 microversion: 2.20 time: 0.349342 Aug 24 10:05:53.294404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 206/417] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:52 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1752 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:53.406581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:53.407339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:53.454242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:53.455043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:53.456264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:53.457104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:53.457736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:53.493355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.035s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:53.706343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:53.715718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-95be1b98-0800-478a-a11f-d212d38c9c04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1701 microversion: 2.1 time: 0.313436 Aug 24 10:05:53.716787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 212/418] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1701 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:53.740346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-458496ba-87cf-42b0-b9ad-d14684ff6598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:53.751945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-458496ba-87cf-42b0-b9ad-d14684ff6598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:53.751945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-458496ba-87cf-42b0-b9ad-d14684ff6598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:53.751945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-458496ba-87cf-42b0-b9ad-d14684ff6598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:53.829640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-458496ba-87cf-42b0-b9ad-d14684ff6598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Going to try to start instance {{(pid=120653) start /opt/stack/nova/nova/compute/api.py:2772}} Aug 24 10:05:53.938608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-458496ba-87cf-42b0-b9ad-d14684ff6598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/action" status: 202 len: 0 microversion: 2.1 time: 0.201153 Aug 24 10:05:53.940074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 207/419] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:05:53 2022] POST /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/action => generated 0 bytes in 205 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:05:53.949123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:53.950220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:53.966540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:53.967154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:53.967708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:53.968865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:53.969463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:53.970027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:54.324640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:54.325625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:54.337585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:54.350715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:54.351984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:54.353374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:54.354696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:54.356150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:54.388353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:54.409657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-041fbb1d-21e2-4910-bb2f-8fc9af76b0ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1710 microversion: 2.1 time: 0.464742 Aug 24 10:05:54.428280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 213/420] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1710 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:54.691925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:54.711856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:54.715577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:54.716808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-063618b1-4f53-4d97-952a-95db166c72e3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1752 microversion: 2.20 time: 0.397204 Aug 24 10:05:54.732453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 208/421] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:54 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1752 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:05:55.435551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:55.437416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:55.443825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:55.444340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:55.444773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:55.445461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:55.446082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:55.446623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:55.468940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a4c1021f-cb31-45b3-ac60-8aefd926af8e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Action: 'create', calling method: .version_select at 0x7fa33da59ee0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-932602861", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL2bc9tP+pTCQSr/XjhRTU/xNXPxNOCaqvWfVYrWfEmumPSj6O2fk6TYGj0o89zLrkCDXyeIGuIftccR0lQrBCgdeMw3uAAnXX3Ylk2VNgB0ZdDSOWmlyWqxg8xyt3zoVA=="}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:55.494456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-a4c1021f-cb31-45b3-ac60-8aefd926af8e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Getting quotas for project 5294ff8dc07444e5b8758dbae2db7fc4. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:05:55.499593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-a4c1021f-cb31-45b3-ac60-8aefd926af8e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Getting quotas for user bea2aecac254429a823b8765feb6c767 and project 5294ff8dc07444e5b8758dbae2db7fc4. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:05:55.536027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a4c1021f-cb31-45b3-ac60-8aefd926af8e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.070782 Aug 24 10:05:55.536665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 209/422] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:05:55 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:05:55.595461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1260261282", "imageRef": "095d1496-a59d-4d26-9e22-c2be06c86584", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-932602861", "networks": [{"uuid": "b2cf8d4b-7b7f-4070-8be1-9619d8709092"}]}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:05:55.608678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:55.621544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2e82898a-36e9-4aec-9050-c78d4799e7f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1710 microversion: 2.1 time: 0.186717 Aug 24 10:05:55.623002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 214/423] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:55 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1710 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:55.700120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] validate_networks() for [('b2cf8d4b-7b7f-4070-8be1-9619d8709092', None, None, None, None, None)] {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:05:55.748341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:55.749244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:55.756615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:55.757113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:55.757750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:55.758534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:55.758983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:55.760967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:55.964669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:55.975549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:55.977245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:55.978402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-96635c80-a5a5-4df3-844b-94c26f58e2bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1752 microversion: 2.20 time: 0.235785 Aug 24 10:05:55.979270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 215/424] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:55 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1752 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:56.177029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:05:56.177461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:05:56.177775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:05:56.178920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:05:56.179425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:05:56.192675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:56.193009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:56.193477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:56.194338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:56.194866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:56.195434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:56.218497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Getting quotas for project 5294ff8dc07444e5b8758dbae2db7fc4. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:05:56.224353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Getting quotas for user bea2aecac254429a823b8765feb6c767 and project 5294ff8dc07444e5b8758dbae2db7fc4. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:05:56.239895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:05:56.242496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='095d1496-a59d-4d26-9e22-c2be06c86584',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:05:56.324210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7ce8bb25-16ff-4a4d-b360-6c9dc921f58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.732540 Aug 24 10:05:56.331548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 210/425] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:05:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 740 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:05:56.352971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-28e95af0-ba67-49d9-9735-a442439eafc6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:56.354000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-28e95af0-ba67-49d9-9735-a442439eafc6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:56.467796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-28e95af0-ba67-49d9-9735-a442439eafc6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:56.480874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-28e95af0-ba67-49d9-9735-a442439eafc6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1258 microversion: 2.1 time: 0.148306 Aug 24 10:05:56.482469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 216/426] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:56 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1258 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:05:56.645601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:56.648434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:56.655850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:56.664550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:56.679298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.023s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:56.680088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:56.680743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:56.681356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:56.882371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:56.892539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0a77ab0d-4caf-4cf5-ae27-17159c1f885d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1714 microversion: 2.1 time: 0.251114 Aug 24 10:05:56.893694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 211/427] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:56 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1714 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:57.004440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:57.005393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:57.020240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:57.020743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:57.021175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:57.027170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:57.027680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:57.028148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:57.232459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:57.247898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:57.249357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:57.250721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fefe4012-201b-4395-8ea6-f727428ad8af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1754 microversion: 2.20 time: 0.252349 Aug 24 10:05:57.251275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 217/428] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1754 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:57.503622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:57.504524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:57.511483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:57.512038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:57.512570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:57.513357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:57.513808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:57.514227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:57.592876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:57.601943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2e834839-005b-4156-b753-b86f34afec7c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1310 microversion: 2.1 time: 0.102694 Aug 24 10:05:57.602480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 212/429] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:57 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1310 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:58.276789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:58.277712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:58.286360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:58.286360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:58.286672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:58.287967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:58.288513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:58.289063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:58.621221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:58.623464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:58.624211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:58.624957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:58.633513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:58.633961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:58.634783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:58.635385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:58.635588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:58.636610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:58.637003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b49b2c6a-6ee5-40f1-9708-1fb97498a955 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.365458 Aug 24 10:05:58.638062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:58.638568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:58.653756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 218/430] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:58 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:05:58.726353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:05:58.735791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8c66bf08-9c59-4ffa-be81-e283e18e316e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1316 microversion: 2.1 time: 0.116707 Aug 24 10:05:58.736664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 213/431] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:58 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1316 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:05:59.663838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:59.665739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:59.684540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:59.686485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:59.687087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:59.689222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:59.693619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:59.694697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:59.757831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:05:59.758764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:05:59.774692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:59.775230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:59.775730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:05:59.781769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:05:59.784816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:05:59.786824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:00.006461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:00.020586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:00.023071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:00.029259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cbee5bad-73e2-4c57-9d93-a31816727975 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.372553 Aug 24 10:06:00.030065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 219/432] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:05:59 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:00.276739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:00.330552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4b7858ae-efab-4b77-a33c-afc25a87c75f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.577050 Aug 24 10:06:00.331462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 214/433] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:05:59 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:00.434468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-56fac0ab-a7a2-4576-adaa-443581e8c912 req-e2e96002-1a15-4b62-8656-2a3fbb811fcc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "987654b5-b3b6-434c-a284-d630ab8ab4ad", "tag": "98183aae-275c-45c1-b9fc-a7d038275f2d"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:00.463215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-56fac0ab-a7a2-4576-adaa-443581e8c912 req-e2e96002-1a15-4b62-8656-2a3fbb811fcc service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:00.464837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-56fac0ab-a7a2-4576-adaa-443581e8c912 req-e2e96002-1a15-4b62-8656-2a3fbb811fcc service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:00.466854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-56fac0ab-a7a2-4576-adaa-443581e8c912 req-e2e96002-1a15-4b62-8656-2a3fbb811fcc service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:00.508504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-56fac0ab-a7a2-4576-adaa-443581e8c912 req-e2e96002-1a15-4b62-8656-2a3fbb811fcc service nova] Creating event network-changed:98183aae-275c-45c1-b9fc-a7d038275f2d for instance 987654b5-b3b6-434c-a284-d630ab8ab4ad on hv-cinder-85427 Aug 24 10:06:00.522011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-56fac0ab-a7a2-4576-adaa-443581e8c912 req-e2e96002-1a15-4b62-8656-2a3fbb811fcc service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.090940 Aug 24 10:06:00.532795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 220/434] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:00.586778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-270075712", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1474637294", "security_groups": [{"name": "tempest-secgroup-smoke-603873071"}], "block_device_mapping_v2": [{"uuid": "678a08dc-270a-4f48-b1e1-7b5273ce4070", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "ac0d3e7c-c9a8-484b-82e2-ae97881bbe2e"}]}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:00.964573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] validate_networks() for [('ac0d3e7c-c9a8-484b-82e2-ae97881bbe2e', None, None, None, None, None)] {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:06:01.059228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:01.060770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:01.067995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:01.069397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:01.069954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:01.070771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:01.071360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:01.071862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:01.372595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=82,nova_api:INSERT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:01.384087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:01.412636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:01.418152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:01.421399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d308fc1a-17af-40ac-9bfe-55486c85779a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.366296 Aug 24 10:06:01.422200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 221/435] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:01.444333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:01.450360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:01.474231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:01.480227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:01.482179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:01.486027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:01.487108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:01.488766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:01.801788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:01.812716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-52266912-57ff-4ea5-835e-63445f0cf6c6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.377939 Aug 24 10:06:01.813112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 222/436] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:01 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:01.957184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:06:01.957893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:06:01.958687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:06:01.959450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:06:01.960031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:06:01.972585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:01.974203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:01.975190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:01.975945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:01.976447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:01.977715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:02.019865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Getting quotas for project dc33f347b062491aaa04b7dc18553cf4. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:06:02.024697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Getting quotas for user 3ccfd2f7e35646b0a60b28b6807cee4b and project dc33f347b062491aaa04b7dc18553cf4. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:06:02.094801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:06:02.098766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='678a08dc-270a-4f48-b1e1-7b5273ce4070',volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:06:02.401197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3d97d690-1ddd-49a0-b035-71288e4b2305 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.820505 Aug 24 10:06:02.401947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 215/437] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:06:00 2022] POST /compute/v2.1/servers => generated 405 bytes in 1822 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:06:02.416029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bceda974-e00e-40fe-9cb2-5fb3be7f0c3a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:02.419214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bceda974-e00e-40fe-9cb2-5fb3be7f0c3a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:02.448291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:02.449151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:02.464333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:02.470519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:02.471125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:02.472343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:02.473063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:02.473818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:02.589128 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bceda974-e00e-40fe-9cb2-5fb3be7f0c3a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:02.607344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bceda974-e00e-40fe-9cb2-5fb3be7f0c3a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1094 microversion: 2.1 time: 0.196815 Aug 24 10:06:02.609577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 223/438] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:02 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1094 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:02.838372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:02.843354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:02.855131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:02.860879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:02.861548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:02.862586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:02.863254 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:02.863683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:02.895551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:02.924016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:02.927106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:02.929892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c847da7f-e12d-4735-97ac-3f3cc2bfbfb3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.486445 Aug 24 10:06:02.931333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 216/439] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:02 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:03.190278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:03.201766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-becfda0c-c0ac-42dd-abdd-96fbc4882c89 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.367846 Aug 24 10:06:03.216074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 224/440] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:02 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:03.633691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:03.635016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:03.641525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:03.642329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:03.642796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:03.643528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:03.643980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:03.644621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:03.829854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:03.851224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dc8b5cc4-26c1-43f8-9cad-40bc86d65071 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1136 microversion: 2.1 time: 0.222960 Aug 24 10:06:03.855329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 217/441] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:03 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1136 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:03.954792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:03.956952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:03.977074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:03.978210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:03.978970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:03.980467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:03.991835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:03.993035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:04.215524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:04.223730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:04.224777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:04.235843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:04.239196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:04.241122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:04.242992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:04.244870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:04.245330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:04.245989 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:04.247729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:04.248352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1b81a5dc-a472-4c80-9830-1cb20c81348d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.297288 Aug 24 10:06:04.261624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 225/442] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:04.505357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:04.518687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-81483a77-235b-4bed-a917-92d2565313d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.294628 Aug 24 10:06:04.519901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 218/443] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:04 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:04.886574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:04.887565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:04.894356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:04.895106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:04.895642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:04.896992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:04.897651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:04.898138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:05.024541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:05.034897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d2894409-111d-436e-8934-153b1a397c23 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.162908 Aug 24 10:06:05.051200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 226/444] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:04 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:05.288082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:05.288082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:05.301385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:05.301980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:05.302912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:05.303614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:05.304095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:05.304720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:05.541410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:05.542155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:05.550559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:05.551257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:05.552238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:05.552645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:05.553421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:05.554238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:05.555330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:05.577679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:05.579308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:05.584740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3fb7b8c8-460a-4e2f-9861-f0e05d3f3870 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.309751 Aug 24 10:06:05.585507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 219/445] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:05.746776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:05.756561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6fac46e1-e907-4a52-a068-5766aee7ba8d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.220461 Aug 24 10:06:05.757149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 227/446] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:05 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:06.058564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:06.059207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:06.065470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:06.065964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:06.066403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:06.067115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:06.067573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:06.068630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:06.155023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:06.164314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bf9371c0-7bf4-4b50-94d3-960fe781a6a3 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.109566 Aug 24 10:06:06.164891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 220/447] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:06 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:06.398742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1140891664", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "networks": [{"uuid": "7ce1e2bd-4840-4443-a463-bed805b1bf48"}]}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:06.613533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:06.614138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:06.620991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:06.621473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:06.621905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:06.622593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:06.623043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:06.623461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:06.771494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:06.781425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:06.782929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:06.784161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4d067bf3-44b0-48d0-a032-71ce4e8a3c36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.174529 Aug 24 10:06:06.784676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 221/448] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:06.795632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:06.795632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:06.798031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:06.798527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:06.798965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:06.799666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:06.802012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:06.802898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:06.943246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:06.952213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1b404c1c-d7a3-4b32-896f-9d76d7677a36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.166335 Aug 24 10:06:06.952892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 222/449] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:06 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:07.189651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:07.190477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:07.197213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:07.197710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:07.198150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:07.198851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:07.199304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:07.199722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:07.205123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:07.286742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:07.296055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-96910e02-d0c5-4935-b2f6-70f9dea49628 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.112796 Aug 24 10:06:07.298462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 223/450] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:07 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:07.809722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:07.810412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:07.816607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:07.817097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:07.818029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:07.818742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:07.819196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:07.819621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:07.982302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:07.992305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:07.993798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:07.994961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-09309e70-fedc-4d82-8850-c86e67329557 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.189312 Aug 24 10:06:07.995517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 224/451] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:06:08.000229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:08.000837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:08.007541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:08.008036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:08.008501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:08.009385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:08.009804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:08.010235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:08.157548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:08.168951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b8dd6cd6-1625-4ebd-b3d9-397d900a9511 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.172133 Aug 24 10:06:08.170328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 225/452] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:07 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:08.319128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:08.320323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:08.327221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:08.327945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:08.328422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:08.329613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:08.330252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:08.330739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:08.400865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:08.410362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-632dcc1b-2115-4f07-a03c-c85deab37318 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.095396 Aug 24 10:06:08.411147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 226/453] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:08 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:09.018039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:09.018750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:09.025317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:09.026124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:09.026643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:09.027607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:09.028170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:09.028659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:09.181810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:09.190862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:09.192394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:09.194148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0708fdee-d61b-45d8-b7e9-06eb02669c53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.180490 Aug 24 10:06:09.195497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 227/454] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:09 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:09.199812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:09.200476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:09.206781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:09.207260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:09.207703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:09.208448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:09.208904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:09.209811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:09.354950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:09.365628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-645cdc71-79aa-45a4-95a2-8b12a5c11672 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.168601 Aug 24 10:06:09.366325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 228/455] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:09 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:09.433581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:09.434192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:09.440195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:09.440685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:09.441129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:09.441828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:09.442291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:09.442721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:09.510675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:09.518810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4ea70908-66d0-4775-9a04-fb9889fe41d0 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.089024 Aug 24 10:06:09.532445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 229/456] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:09 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:10.217691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:10.218393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:10.225207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:10.225626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:10.226059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:10.226758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:10.227206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:10.227625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:10.387350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:10.398928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:10.400461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:10.401631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8e6bdef8-8d5b-4e8f-91b5-e645f71247d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.188244 Aug 24 10:06:10.412473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 230/457] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:10 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:06:10.412473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:10.412473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:10.416097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:10.416665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:10.417038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:10.417844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:10.418300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:10.418742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:10.563520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:10.573159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-47de990c-aedd-4c3f-8d7a-719c4f72b53b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.169099 Aug 24 10:06:10.573745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 231/458] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:10 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:10.578381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:10.578976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:10.586224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:10.586819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:10.587339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:10.588097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:10.588589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:10.589059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:10.665619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:10.674572 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-756e08bb-e032-4b47-a841-db08f2620b01 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.099670 Aug 24 10:06:10.675483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 232/459] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:10 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:11.425762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:11.426573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:11.433107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:11.433664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:11.434107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:11.434886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:11.435363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:11.435769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:11.571678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:11.580699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:11.582211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:11.583393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5e2c47c1-e0dd-4873-92cb-a1cc59be926f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.161566 Aug 24 10:06:11.583946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 233/460] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:11 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:06:11.604171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:11.604790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:11.613378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:11.616782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:11.618017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:11.619282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:11.620018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:11.620697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:11.775297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:11.783415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7a70e08b-6b2d-4351-bedf-11dd535d700c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.186253 Aug 24 10:06:11.784004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 234/461] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:11 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:11.789210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:11.789957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:11.797255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:11.797752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:11.798195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:11.798902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:11.799361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:11.799791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:11.875363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:11.884736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-00d77ba6-5a37-4bb8-bab5-48a01ca5d574 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.099631 Aug 24 10:06:11.889931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 235/462] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:11 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:12.594192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:12.608187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:12.608802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:12.610625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:12.611367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=16 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:12.619035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:12.619545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:12.620556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:12.621279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:12.621748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:12.622177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:12.730876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] validate_networks() for [('7ce1e2bd-4840-4443-a463-bed805b1bf48', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:06:12.760657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:12.779743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:12.782172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:12.787106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-53d7c602-e71a-47dc-bf6d-1ca0e52bb3d3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.182983 Aug 24 10:06:12.790626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 236/463] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:12 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:06:12.806556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:12.807176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:12.813526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:12.814573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:12.814573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:12.815331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:12.815827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:12.816278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:12.969401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:12.979946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-141de860-7f4b-43a7-b3f1-d6a6b913287c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.177969 Aug 24 10:06:12.992314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 237/464] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:12 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:12.992314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:12.992314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:12.998307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:12.999552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:13.000025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:13.001181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:13.001663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:13.002477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:13.083539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:13.091875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-69b9cf84-8534-4542-a1f6-f244711d1fcc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.109653 Aug 24 10:06:13.093394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 238/465] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:12 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:13.138882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:06:13.139390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:06:13.139995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:06:13.140886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:06:13.141909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:06:13.167384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:13.167888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:13.169276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:13.170501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:13.171151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:13.171673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:13.202256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Getting quotas for project f1eac6259f65441d9abbabe2a04afad1. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:06:13.208275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Getting quotas for user 4ce9eb7417a7482cb9cdaed16cb6aeb9 and project f1eac6259f65441d9abbabe2a04afad1. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:06:13.220837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:06:13.224010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:06:13.277142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d5840192-3491-4257-bfe4-91b802914da7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 6.882211 Aug 24 10:06:13.278189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 228/466] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:06:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 6884 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:06:13.291614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-960a34c1-8909-4f29-9055-2e96d317c2e5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:13.292596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-960a34c1-8909-4f29-9055-2e96d317c2e5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:13.593581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-960a34c1-8909-4f29-9055-2e96d317c2e5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:13.601658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-960a34c1-8909-4f29-9055-2e96d317c2e5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1219 microversion: 2.1 time: 0.314394 Aug 24 10:06:13.612110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 239/467] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:13 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1219 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:13.814048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:13.814658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:13.824985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:13.825477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:13.825989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:13.826881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:13.827374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:13.827915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:13.997511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:14.005863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:14.006825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:14.008004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:14.009710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:14.010980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1a684d5d-8a21-44b8-a2de-ee47942d2da7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.201696 Aug 24 10:06:14.014199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:14.014933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:14.016080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:14.017007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:14.017655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:14.018281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:14.027964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 229/468] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:14.122430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:14.123336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:14.130247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:14.131093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:14.131527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:14.132270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:14.132743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:14.133153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:14.171456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:14.175053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:14.184107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9788c9bb-1fdd-4aa7-8912-e4ce8e1c4f47 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.180477 Aug 24 10:06:14.184724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 240/469] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:13 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:14.220608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:14.233072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a17fcea5-ebea-4463-a951-0347b417e9a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.118266 Aug 24 10:06:14.233770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 230/470] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:14 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:14.638424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:14.639346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:14.651865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:14.652571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:14.653085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:14.653983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:14.654469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:14.655228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:14.733381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:14.741830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d697036e-95d7-48b1-8056-ecac50f635b7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1223 microversion: 2.1 time: 0.118231 Aug 24 10:06:14.745049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 241/471] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:14 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1223 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:15.038193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:15.038773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:15.045252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:15.045731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:15.046159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:15.046878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:15.047326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:15.047742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:15.187656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:15.197412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:15.199315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:15.200662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7665c235-6356-414a-8f34-3287c99d76ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.167914 Aug 24 10:06:15.201228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 231/472] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:15.219356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:15.219965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:15.226842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:15.227334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:15.227921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:15.228646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:15.229366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:15.229820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:15.255010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:15.255628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:15.261929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:15.262415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:15.262848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:15.263551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:15.264004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:15.264646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:15.351760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:15.362608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ad3fcce5-ba65-45ee-8c50-17c8ccb4443e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.111380 Aug 24 10:06:15.378489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 232/473] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:15 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:15.419682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:15.427511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3c4c8e2c-e426-45f7-b4c5-03fee77e0192 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.212368 Aug 24 10:06:15.428384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 242/474] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:15 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:15.762834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:15.763797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:15.770635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:15.771132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:15.771571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:15.772370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:15.772830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:15.774325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:15.841090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:15.851816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-20b9b7ed-227c-4a9a-971c-bede4695eaab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1223 microversion: 2.1 time: 0.092598 Aug 24 10:06:15.852501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 233/475] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:15 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1223 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:16.226582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:16.227202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:16.234117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:16.234616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:16.235056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:16.235758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:16.236251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:16.236676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:16.389610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:16.391323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:16.392372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:16.400586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:16.400781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:16.401811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:16.402246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:16.404311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6c318196-41d4-438f-a178-06cba4b022a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.181884 Aug 24 10:06:16.405425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 243/476] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:16 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:16.412844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:16.415605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:16.417662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:16.420310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:16.455061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:16.455790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:16.463826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:16.464350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:16.464788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:16.465601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:16.466050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:16.466552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:16.508690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:16.521880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-719ca9c6-a51f-4afa-9af7-cf5c973c0b36 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.135072 Aug 24 10:06:16.522666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 234/477] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:16 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:16.621497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:16.629470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-193dd0b1-ed40-4102-b132-419d167d7067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.179749 Aug 24 10:06:16.637250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 244/478] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:16 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:16.874984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:16.875596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:16.883435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:16.883924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:16.884459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:16.885163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:16.885611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:16.886031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:16.954290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:16.962788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e541c80c-fd87-461d-ad86-3c8138ad1089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1223 microversion: 2.1 time: 0.091683 Aug 24 10:06:16.963648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 235/479] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:16 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1223 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:17.429356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:17.429986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:17.437480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:17.437981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:17.438428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:17.439139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:17.439600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:17.440032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:17.542593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:17.543286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:17.550923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:17.552076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:17.552583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:17.553412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:17.553887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:17.554367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:17.586335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:17.595658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:17.597209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:17.598925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4633e756-464a-416b-8a3e-d3b1500a9e69 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.173562 Aug 24 10:06:17.599479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 245/480] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:17.627914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:17.635606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0dc9d2c2-a9d6-447a-823d-e6508b154fe6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.096793 Aug 24 10:06:17.636190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 236/481] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:17 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:17.649757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:17.650405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:17.656818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:17.657323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:17.657751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:17.658461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:17.658914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:17.659387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:17.820904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:17.828740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0aa1cfa6-c4ff-4066-8bd8-31a5ab7c128b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.182762 Aug 24 10:06:17.829580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 246/482] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:17 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:17.987049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:17.987664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:17.993980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:17.994472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:17.994909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:17.995611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:17.996085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:17.996512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:18.060630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:18.062939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:18.069937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d51fd17d-dcf0-46cf-8612-0e946ac625c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1223 microversion: 2.1 time: 0.087209 Aug 24 10:06:18.070504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 237/483] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:17 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1223 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:18.622631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:18.623282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:18.629912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:18.630402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:18.630841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:18.631545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:18.631999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:18.632449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:18.657361 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:18.657973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:18.664656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:18.665129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:18.665558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:18.666242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:18.666690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:18.667102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:18.737535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:18.748772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-503cadeb-8f1d-4b41-b03d-6df1097b4cfa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.095379 Aug 24 10:06:18.764570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 238/484] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:18 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:18.801653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:18.811489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:18.813985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:18.815531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c0b68468-4f92-4126-9b70-770f7fce74b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.196910 Aug 24 10:06:18.816328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 247/485] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:18 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:18.849601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:18.850201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:18.856367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:18.856843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:18.857280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:18.857963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:18.858404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:18.858822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:18.996776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:19.005080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-66eda2e6-1f9c-41c2-b802-5b1234b428ac tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.158595 Aug 24 10:06:19.005744 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 239/486] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:18 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:19.092612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:19.093127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:19.099471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:19.099948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:19.100407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:19.101096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:19.101531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:19.101948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:19.165420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:19.173325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-aa9d9b86-cf18-45e9-9dfe-91954080751d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1223 microversion: 2.1 time: 0.084687 Aug 24 10:06:19.173891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 248/487] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:19 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1223 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:19.773887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:19.774501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:19.780799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:19.781291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:19.781727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:19.782426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:19.782880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:19.783525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:19.839272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:19.839975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:19.846542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:19.847002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:19.847660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:19.848411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:19.848657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:19.848916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:19.849347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:19.856766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a8f9fa19-0852-493f-bac6-3293bc99d6ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.088188 Aug 24 10:06:19.868122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 240/488] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:19 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:19.982202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:19.991506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:19.993039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:19.994199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ccdfccbe-7bcf-412b-aa3a-0616794e49f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.158817 Aug 24 10:06:19.994750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 249/489] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:06:20.026976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:20.027554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:20.033501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:20.033996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:20.034433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:20.035121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:20.035578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:20.036004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:20.171834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:20.179634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cab29913-5dec-43c6-b608-1b84834f512e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.156611 Aug 24 10:06:20.180221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 241/490] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:20 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:20.195060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:20.195711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:20.201817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:20.202302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:20.202740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:20.203527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:20.203902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:20.204375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:20.266688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:20.274414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d12f9f11-030e-483d-b358-733a200c911b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1271 microversion: 2.1 time: 0.083724 Aug 24 10:06:20.274992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 250/491] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:20 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1271 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:20.879446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:20.880006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:20.888622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:20.889119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:20.889596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:20.890294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:20.890790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:20.891292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:20.967123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:20.976323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a0de3ec5-e0eb-4933-aab2-0a1d46356671 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1192 microversion: 2.1 time: 0.101466 Aug 24 10:06:20.976963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 242/492] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:20 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1192 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:21.017367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:21.017951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:21.024605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:21.025096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:21.025540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:21.026233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:21.026722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:21.027153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:21.158424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:21.167465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:21.168984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:21.170306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-08237c57-97bd-47a6-a3b7-4288bf933eea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.156997 Aug 24 10:06:21.170851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 251/493] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:21 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:21.199957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:21.200606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:21.206496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:21.206979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:21.207898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:21.208607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:21.209058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:21.209479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:21.296188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:21.296949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:21.305473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:21.306309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:21.306973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:21.308009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:21.308740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:21.309553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:21.439325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:21.453110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ffb6feef-e1ad-4a4a-b68c-0fa09229aad7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1271 microversion: 2.1 time: 0.161142 Aug 24 10:06:21.453913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 252/494] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:21 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1271 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:21.466120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:21.477235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-14217fa5-e7af-48f1-9d34-f27269b31f93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.280997 Aug 24 10:06:21.478292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 243/495] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:21 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:21.998429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:21.999262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:22.008878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:22.011542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:22.012466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:22.013615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:22.014253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:22.014864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:22.106618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:22.115430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7b11f887-7dc0-487e-9c9a-7084c0afe8e5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1210 microversion: 2.1 time: 0.121562 Aug 24 10:06:22.115997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 253/496] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:21 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1210 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:22.196305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:22.197296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:22.200935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:22.208148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:22.208922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:22.209651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:22.210564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:22.211262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:22.211895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:22.455995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:22.474062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:22.476061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:22.477600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bdb90d32-6a32-4877-bca8-b060e77f13c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.288548 Aug 24 10:06:22.478648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:22.479284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:22.484095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 244/497] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:22 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:22.490138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:22.490631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:22.493257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:22.495132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:22.499005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:22.500281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:22.501037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:22.501565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:22.512197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:22.512936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:22.513716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:22.514626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:22.515385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:22.516514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:22.838665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:22.849726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d2bbc27a-5191-468a-a24f-e4ebffc7b1db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1319 microversion: 2.1 time: 0.375644 Aug 24 10:06:22.850565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 254/498] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:22 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1319 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:22.891925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:22.900150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-80033570-fae2-4734-923b-946b58582f87 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.405453 Aug 24 10:06:22.900882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 245/499] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:22 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:23.139918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:23.141266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:23.147933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.148946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.149404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.150182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.150972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.151388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.405187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:23.415179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d1ba9c69-1b2e-4a91-ab15-cc51157885fd tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.281551 Aug 24 10:06:23.416195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 255/500] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:23 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:23.487096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-5c981f20-fe59-4fa6-8ba3-30c86ce93f25 req-fc9244fd-6490-4da0-9cfc-65aada47e58b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0c3cf0b5-40f3-40f9-83ec-6de8d63aca02", "tag": "cac7341d-d892-4b93-a35e-a6dc241272a3"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:23.496288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:23.498054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=6 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:23.500681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5c981f20-fe59-4fa6-8ba3-30c86ce93f25 req-fc9244fd-6490-4da0-9cfc-65aada47e58b service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.501276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5c981f20-fe59-4fa6-8ba3-30c86ce93f25 req-fc9244fd-6490-4da0-9cfc-65aada47e58b service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.501811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5c981f20-fe59-4fa6-8ba3-30c86ce93f25 req-fc9244fd-6490-4da0-9cfc-65aada47e58b service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.511075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:23.511777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:23.521308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.521815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.522257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.525013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.525570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.526082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.531759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-5c981f20-fe59-4fa6-8ba3-30c86ce93f25 req-fc9244fd-6490-4da0-9cfc-65aada47e58b service nova] Creating event network-changed:cac7341d-d892-4b93-a35e-a6dc241272a3 for instance 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 on hv-cinder-85427 Aug 24 10:06:23.539550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-5c981f20-fe59-4fa6-8ba3-30c86ce93f25 req-fc9244fd-6490-4da0-9cfc-65aada47e58b service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057284 Aug 24 10:06:23.540754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 246/501] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:23.598531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:23.624275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-f160f831-767a-4a71-8805-47c0cce09176 req-b5e83cbc-80dc-490f-bfd4-6212f0cf1480 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "350e1445-9336-439c-b146-ee4c74211b54", "tag": "e7af2444-9e35-49f8-aeab-b94839769f32"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:23.635368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f160f831-767a-4a71-8805-47c0cce09176 req-b5e83cbc-80dc-490f-bfd4-6212f0cf1480 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.635978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f160f831-767a-4a71-8805-47c0cce09176 req-b5e83cbc-80dc-490f-bfd4-6212f0cf1480 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.636505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f160f831-767a-4a71-8805-47c0cce09176 req-b5e83cbc-80dc-490f-bfd4-6212f0cf1480 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.649794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-f160f831-767a-4a71-8805-47c0cce09176 req-b5e83cbc-80dc-490f-bfd4-6212f0cf1480 service nova] Creating event network-changed:e7af2444-9e35-49f8-aeab-b94839769f32 for instance 350e1445-9336-439c-b146-ee4c74211b54 on hv-cinder-85427 Aug 24 10:06:23.655492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-f160f831-767a-4a71-8805-47c0cce09176 req-b5e83cbc-80dc-490f-bfd4-6212f0cf1480 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036093 Aug 24 10:06:23.656469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 247/502] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:23.774144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:23.783832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:23.785429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:23.786854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-777deba9-4ba7-4b65-aa51-ac30f1332c95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.279749 Aug 24 10:06:23.787415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 256/503] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:23 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:23.873526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:23.874235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:23.883528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.884022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.884519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.885230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.885716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.886143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.923596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:23.924344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:23.932879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:23.934470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.939644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.940164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.940925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:23.941918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:23.942533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:23.967097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:24.140009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:24.154913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dc6b47b2-db86-4af0-bf10-7a382b84acac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1319 microversion: 2.1 time: 0.283875 Aug 24 10:06:24.156909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 248/504] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:23 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1319 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:24.167998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-602ae0ac-a457-4ddb-8b7c-de40098cb2b6 req-79096dde-31f0-4c91-8c50-6dba5e1406b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "987654b5-b3b6-434c-a284-d630ab8ab4ad", "name": "network-vif-plugged", "status": "completed", "tag": "98183aae-275c-45c1-b9fc-a7d038275f2d"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:24.183270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-602ae0ac-a457-4ddb-8b7c-de40098cb2b6 req-79096dde-31f0-4c91-8c50-6dba5e1406b9 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:24.184315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-602ae0ac-a457-4ddb-8b7c-de40098cb2b6 req-79096dde-31f0-4c91-8c50-6dba5e1406b9 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:24.184911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-602ae0ac-a457-4ddb-8b7c-de40098cb2b6 req-79096dde-31f0-4c91-8c50-6dba5e1406b9 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:24.187933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:24.197470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-602ae0ac-a457-4ddb-8b7c-de40098cb2b6 req-79096dde-31f0-4c91-8c50-6dba5e1406b9 service nova] Creating event network-vif-plugged:98183aae-275c-45c1-b9fc-a7d038275f2d for instance 987654b5-b3b6-434c-a284-d630ab8ab4ad on hv-cinder-85427 Aug 24 10:06:24.200610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c345bc6a-bcca-415a-9b85-39c78a51baa2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.281381 Aug 24 10:06:24.201462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 257/505] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:23 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:24.214167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-602ae0ac-a457-4ddb-8b7c-de40098cb2b6 req-79096dde-31f0-4c91-8c50-6dba5e1406b9 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055595 Aug 24 10:06:24.216527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 249/506] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:24.436720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:24.437342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:24.443516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:24.444005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:24.444576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:24.449378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:24.449845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:24.450280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:24.591240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:24.599352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-befb5bd3-ded2-4bc4-81a7-09797bad4787 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.166637 Aug 24 10:06:24.599917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 258/507] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:24 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:24.811071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:24.811874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:24.818123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:24.818606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:24.819042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:24.819729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:24.820225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:24.820653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:24.953365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:24.962630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:24.964136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:24.965274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e80edc35-c1bb-419f-8448-482fd7f06df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.158973 Aug 24 10:06:24.965822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 250/508] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:24 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:25.176987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:25.177627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:25.183810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:25.184364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:25.184798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:25.185480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:25.185926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:25.186346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:25.224131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:25.224734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:25.231089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:25.231604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:25.232035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:25.232754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:25.233203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:25.233615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:25.319321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:25.327075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4a1e6f2a-7efb-4203-a31a-8b64ecfc3c8b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1319 microversion: 2.1 time: 0.153879 Aug 24 10:06:25.327645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 259/509] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:25 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:25.368234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:25.375802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c037da48-2351-4174-845f-6b5e93b76255 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.155458 Aug 24 10:06:25.376397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 251/510] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:25 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:25.620388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:25.620905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:25.628280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:25.629435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:25.629872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:25.630566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:25.631070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:25.631496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:25.759039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:25.766677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-afeda249-8300-41e0-92e5-465f1ec1a829 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.150257 Aug 24 10:06:25.767234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 260/511] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:25 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:26.003406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:26.004227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:26.011154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:26.011861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:26.012499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:26.013354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:26.013965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:26.014543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:26.172989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:26.182543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:26.184086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:26.185248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-185a7087-44e3-472b-bbbd-37644ead8d9a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.188184 Aug 24 10:06:26.185805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 252/512] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:25 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:26.349682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:26.350232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:26.356412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:26.356903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:26.357345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:26.358043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:26.358500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:26.358929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:26.397148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:26.397758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:26.403750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:26.404272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:26.404712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:26.405409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:26.405859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:26.406281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:26.492176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:26.500255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4313374e-5371-4efe-81ff-b0b2167992f4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1319 microversion: 2.1 time: 0.155032 Aug 24 10:06:26.501178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 261/513] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:26 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:26.540546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:26.548917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6d33bc82-e310-4d6d-a3cc-87251b4d227b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.156046 Aug 24 10:06:26.549483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 253/514] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:26 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:26.786991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:26.787604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:26.793695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:26.794180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:26.794620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:26.795309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:26.795761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:26.796230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:26.937707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:26.945897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d1896dd5-1c0f-4d51-8726-15968ceb0bbf tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.162950 Aug 24 10:06:26.946455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 262/515] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:26 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:27.208267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:27.209475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:27.215372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:27.216365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:27.216816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:27.217527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:27.218048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:27.218476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:27.401600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:27.411155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:27.412710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:27.413889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-256a0d53-3c3a-45ea-9e8f-14863c6d001b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.209678 Aug 24 10:06:27.414451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 254/516] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:27 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:27.521985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:27.522606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:27.528534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:27.529022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:27.529845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:27.530540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:27.531006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:27.531528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:27.569501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:27.570110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:27.576358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:27.576840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:27.577269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:27.577963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:27.578411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:27.578830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:27.691716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:27.700766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3e69882c-8806-41a5-b4a4-0f71fe151062 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1319 microversion: 2.1 time: 0.182611 Aug 24 10:06:27.701358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 263/517] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:27 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1319 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:27.789626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:27.798157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cde2b4fa-115c-4aa3-8381-6e9b3a6049cb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.232660 Aug 24 10:06:27.798803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 255/518] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:27 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:27.967354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:27.968918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:27.976366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:27.976872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:27.977327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:27.978060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:27.979107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:27.979707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:28.195182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:28.203607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d7ad90f4-43e8-42a1-a2fa-7bfd8330b81d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.240257 Aug 24 10:06:28.204227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 264/519] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:27 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:28.290201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-fae48cd5-6ffb-430c-8fea-2d1a94c2ffee req-28acb0fd-14b3-47e3-8c9d-c5f27d63925d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0c3cf0b5-40f3-40f9-83ec-6de8d63aca02", "name": "network-vif-plugged", "status": "completed", "tag": "cac7341d-d892-4b93-a35e-a6dc241272a3"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:28.300256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-fae48cd5-6ffb-430c-8fea-2d1a94c2ffee req-28acb0fd-14b3-47e3-8c9d-c5f27d63925d service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:28.301317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-fae48cd5-6ffb-430c-8fea-2d1a94c2ffee req-28acb0fd-14b3-47e3-8c9d-c5f27d63925d service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:28.301711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-fae48cd5-6ffb-430c-8fea-2d1a94c2ffee req-28acb0fd-14b3-47e3-8c9d-c5f27d63925d service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:28.314663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-fae48cd5-6ffb-430c-8fea-2d1a94c2ffee req-28acb0fd-14b3-47e3-8c9d-c5f27d63925d service nova] Creating event network-vif-plugged:cac7341d-d892-4b93-a35e-a6dc241272a3 for instance 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 on hv-cinder-85427 Aug 24 10:06:28.319454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-fae48cd5-6ffb-430c-8fea-2d1a94c2ffee req-28acb0fd-14b3-47e3-8c9d-c5f27d63925d service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033134 Aug 24 10:06:28.321512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 256/520] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:28.437803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:28.438556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:28.444546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:28.445032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:28.445471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:28.446376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:28.446832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:28.447261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:28.610940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:28.619451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:28.621139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:28.622284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b72c5e7d-40c6-4b31-ad66-ca6e74e92e76 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.188751 Aug 24 10:06:28.623085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 265/521] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:28 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:28.723108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:28.723797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:28.730212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:28.730684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:28.731110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:28.731785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:28.732256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:28.732670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:28.736554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:28.818982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:28.819586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:28.825883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:28.826354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:28.826783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:28.827662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:28.828143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:28.828557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:28.866820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:28.877962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8f243eeb-5326-4cfa-99b6-7a0ab07717d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1319 microversion: 2.1 time: 0.159597 Aug 24 10:06:28.879359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 257/522] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:28 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1319 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:28.968814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:28.977732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-36b95059-e187-4344-85b0-4007dc450710 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1358 microversion: 2.1 time: 0.163021 Aug 24 10:06:28.978665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 266/523] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:28 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1358 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:29.225164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:29.225790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:29.232613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:29.233092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:29.233523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:29.234243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:29.234689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:29.235184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:29.368698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:29.376326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d3e14a68-4ae2-4d67-a097-4b62a9875ab7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.155190 Aug 24 10:06:29.376887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 258/524] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:29 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:29.645409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:29.645951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:29.652181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:29.652674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:29.653109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:29.653791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:29.654253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:29.654677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:29.781715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:29.791322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:29.792877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:29.794046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-43afdba9-2af6-4f22-b09d-a88b125d36d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.152672 Aug 24 10:06:29.794606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 267/525] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:29.899994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:29.900559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:29.906513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:29.906991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:29.907421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:29.908132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:29.908583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:29.909003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:29.999004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:29.999714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:30.005763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.006251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.006692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.007417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.007874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.008329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.054015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:30.062147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0dc4056e-86fd-44ee-9db6-2d8e3908a312 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1319 microversion: 2.1 time: 0.166312 Aug 24 10:06:30.063189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 259/526] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:29 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1319 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:30.177784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:30.186378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f3dd56ca-da82-4a18-a86b-24b3fb4a6c01 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1546 microversion: 2.1 time: 0.191778 Aug 24 10:06:30.187200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 268/527] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:29 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1546 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:30.210638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:30.211269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:30.218267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.218267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.218550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.219056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.219498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.219915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.336186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-2e6d4376-12b1-46f4-a560-3eb427714a56 req-5b717b4d-e11f-4907-837a-7cab9eedfb54 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1", "name": "network-vif-plugged", "status": "completed", "tag": "1b9069bd-2630-4b65-9e91-d7695e3b6a8c"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:30.346717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-2e6d4376-12b1-46f4-a560-3eb427714a56 req-5b717b4d-e11f-4907-837a-7cab9eedfb54 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.347231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-2e6d4376-12b1-46f4-a560-3eb427714a56 req-5b717b4d-e11f-4907-837a-7cab9eedfb54 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.347670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-2e6d4376-12b1-46f4-a560-3eb427714a56 req-5b717b4d-e11f-4907-837a-7cab9eedfb54 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.360250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-2e6d4376-12b1-46f4-a560-3eb427714a56 req-5b717b4d-e11f-4907-837a-7cab9eedfb54 service nova] Creating event network-vif-plugged:1b9069bd-2630-4b65-9e91-d7695e3b6a8c for instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 on hv-cinder-85427 Aug 24 10:06:30.373557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-2e6d4376-12b1-46f4-a560-3eb427714a56 req-5b717b4d-e11f-4907-837a-7cab9eedfb54 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041696 Aug 24 10:06:30.377065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:30.381064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 269/528] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:30.397189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-18b18374-af49-4e51-936d-a1085d1a9af7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1546 microversion: 2.1 time: 0.192682 Aug 24 10:06:30.401055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:30.401896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:30.415661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.416194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.416831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.417544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.418033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.418466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.422651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 260/529] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:30 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1546 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:30.427907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:30.432735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:30.433502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Searching by: {'deleted': False, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4'} {{(pid=120654) get_all /opt/stack/nova/nova/compute/api.py:3004}} Aug 24 10:06:30.440224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.440706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.441665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.442391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.442843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.443268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.535150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.multi_cell_list [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120654) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Aug 24 10:06:30.555165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:30.563523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-da0e0f18-da2d-4c49-90fa-5baef8083086 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.167337 Aug 24 10:06:30.564345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 270/530] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:30 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:30.635709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-deb4e885-d9db-4028-a766-59ae170bebeb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.211655 Aug 24 10:06:30.636312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 261/531] 10.222.0.86 () {58 vars in 1125 bytes} [Wed Aug 24 10:06:30 2022] GET /compute/v2.1/servers => generated 343 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:30.647886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:30.648553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:30.654434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.654911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.655344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.656022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.656519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.656936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.788899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:30.796728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0ad82a5e-b2eb-4b8d-b51c-f884ecd5b924 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1546 microversion: 2.1 time: 0.152621 Aug 24 10:06:30.797278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 271/532] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:30 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1546 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:30.817816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:30.818429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:30.824508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.825002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.825469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.826168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:30.826626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:30.827051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:30.977470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:30.987792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:30.989952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:30.991752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1345d15f-eddb-4355-865a-dfca205567cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.177953 Aug 24 10:06:30.992465 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 262/533] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:31.083802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:31.084473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:31.097326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:31.097897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:31.098391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:31.099269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:31.100160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:31.100939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:31.242390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:31.250114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-37674aa2-a6e8-46d0-83e9-c64c4295acd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1509 microversion: 2.1 time: 0.170680 Aug 24 10:06:31.250669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 272/534] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:31 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1509 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:31.589310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:31.590053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:31.597108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:31.597938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:31.598770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:31.599691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:31.600937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:31.601605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:31.766239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:31.775102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9831f518-de3e-428d-a5cf-dc105614a610 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.192502 Aug 24 10:06:31.776093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 263/535] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:31 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:32.014905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:32.015558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:32.021705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:32.022119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:32.022552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:32.023245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:32.023696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:32.024144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:32.160943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:32.169926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:32.171433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:32.172641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1fbb0bd7-6972-4ebb-9860-a2fee62e448d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.161854 Aug 24 10:06:32.173196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 273/536] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:32 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:32.397298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2fcafd14-6882-45b4-9797-9410b528c7de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "53ed8765-f72f-48b6-8b3c-c9f975174c7e"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:32.399662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2fcafd14-6882-45b4-9797-9410b528c7de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:32.405980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fcafd14-6882-45b4-9797-9410b528c7de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:32.406474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fcafd14-6882-45b4-9797-9410b528c7de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:32.406912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fcafd14-6882-45b4-9797-9410b528c7de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:32.407628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fcafd14-6882-45b4-9797-9410b528c7de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:32.408102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fcafd14-6882-45b4-9797-9410b528c7de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:32.408532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fcafd14-6882-45b4-9797-9410b528c7de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:32.799296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:32.799951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:32.806919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:32.807711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:32.808252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:32.809312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:32.809991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:32.810616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:32.942552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:32.946153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:32.951756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b72a0c00-0b22-4558-8d0f-22fdfa310892 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.156726 Aug 24 10:06:32.952581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 274/537] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:32 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:33.197046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:33.197727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:33.203668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:33.204262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:33.204696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:33.205385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:33.205838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:33.206261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:33.343271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:33.352504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:33.354252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:33.355403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-98a82225-e1de-4c2c-bc21-34c0ffda2198 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.162583 Aug 24 10:06:33.355951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 275/538] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:33.973699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:33.974314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:33.980540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:33.981025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:33.981468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:33.982146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:33.982603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:33.983027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:34.114146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:34.121759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b54a6b22-da7e-412e-a873-ad996b42df49 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.152594 Aug 24 10:06:34.122309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 276/539] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:33 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:34.379574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:34.380282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:34.386022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:34.386506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:34.386945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:34.387628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:34.388107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:34.388531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:34.520522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:34.529409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:34.530840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:34.532001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ecafd19c-1979-4923-b3f1-785ee63c501c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1810 microversion: 2.20 time: 0.157037 Aug 24 10:06:34.532598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 277/540] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:34 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1810 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:06:35.119175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5a0490f5-7929-45e9-bdb8-ae1e90fb0c52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "57062e38-4dec-4132-969b-b4e85fc3b9ac"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:35.121563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5a0490f5-7929-45e9-bdb8-ae1e90fb0c52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:35.127544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a0490f5-7929-45e9-bdb8-ae1e90fb0c52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:35.128092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a0490f5-7929-45e9-bdb8-ae1e90fb0c52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:35.128533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a0490f5-7929-45e9-bdb8-ae1e90fb0c52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:35.129222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a0490f5-7929-45e9-bdb8-ae1e90fb0c52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:35.129679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a0490f5-7929-45e9-bdb8-ae1e90fb0c52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:35.130102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a0490f5-7929-45e9-bdb8-ae1e90fb0c52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:40.475431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:42.403398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=41 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:42.480548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=49 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:44.570263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2fcafd14-6882-45b4-9797-9410b528c7de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "POST /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 12.177239 Aug 24 10:06:44.571072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 264/541] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:06:32 2022] POST /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments => generated 194 bytes in 12179 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:44.575977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:44.576789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:44.583912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:44.584677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:44.585251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:44.586564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:44.587155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:44.587711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:44.604214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5a0490f5-7929-45e9-bdb8-ae1e90fb0c52 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 9.483160 Aug 24 10:06:44.605342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 278/542] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:06:35 2022] POST /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 194 bytes in 9491 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:06:44.613775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:44.614724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:44.642167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:44.642167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:44.642167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:44.642871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:44.643497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:44.643997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:44.774438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:44.785575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ecc9e11f-f12e-488b-b633-609f6a0924f9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.213197 Aug 24 10:06:44.786448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 265/543] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:44 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:44.815524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:44.824327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:44.825839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:44.827048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4f047a1d-d6c3-4922-a01c-b1ba77be208d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1797 microversion: 2.20 time: 0.219876 Aug 24 10:06:44.827556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 279/544] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:06:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1797 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:06:45.806332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:45.806945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:45.813598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:45.814075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:45.815193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:45.815925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:45.816394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:45.816813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:45.944382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:45.951971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-98d1a732-cd4a-4450-ac49-df652e73905f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.149459 Aug 24 10:06:45.952573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 266/545] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:45 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:46.972215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:46.972831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:46.979495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:46.980016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:46.980640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:46.981189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:46.981644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:46.982075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:47.118188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:47.126434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ffea081e-9e9b-4e2d-999a-3199a252a5de tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.158611 Aug 24 10:06:47.132450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 280/546] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:46 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:48.149368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:48.150148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:48.158796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:48.159300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:48.159896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:48.160661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:48.161261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:48.161636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:48.247409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-d835d607-7ad2-47ad-9227-7aca72097011 req-3a1ee406-40fe-49bf-8fda-70226306bf26 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "350e1445-9336-439c-b146-ee4c74211b54", "name": "network-vif-plugged", "status": "completed", "tag": "e7af2444-9e35-49f8-aeab-b94839769f32"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:48.258758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-d835d607-7ad2-47ad-9227-7aca72097011 req-3a1ee406-40fe-49bf-8fda-70226306bf26 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:48.259236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-d835d607-7ad2-47ad-9227-7aca72097011 req-3a1ee406-40fe-49bf-8fda-70226306bf26 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:48.259877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-d835d607-7ad2-47ad-9227-7aca72097011 req-3a1ee406-40fe-49bf-8fda-70226306bf26 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:48.274624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-d835d607-7ad2-47ad-9227-7aca72097011 req-3a1ee406-40fe-49bf-8fda-70226306bf26 service nova] Creating event network-vif-plugged:e7af2444-9e35-49f8-aeab-b94839769f32 for instance 350e1445-9336-439c-b146-ee4c74211b54 on hv-cinder-85427 Aug 24 10:06:48.281982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-d835d607-7ad2-47ad-9227-7aca72097011 req-3a1ee406-40fe-49bf-8fda-70226306bf26 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037737 Aug 24 10:06:48.283801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 281/547] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:06:48.311363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:48.320537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f9ced1c4-4c4c-47c6-84ce-81ae43794f8e tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.175225 Aug 24 10:06:48.321385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 267/548] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:48 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:06:49.342205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:49.342908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:49.349523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:49.350008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:49.350452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:49.351141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:49.351600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:49.352029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:49.481056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:49.489225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e0c5dfd1-dfc3-485d-8710-5ca9b21354e9 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1265 microversion: 2.1 time: 0.151232 Aug 24 10:06:49.489790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 282/549] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:49 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1265 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:49.723524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-26437bff-3b05-479a-9bc3-fe1e4fa8bf57 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c33defcd-3e83-4c15-bd33-d4758c15058f", "device": "/dev/sdb"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:49.726473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-26437bff-3b05-479a-9bc3-fe1e4fa8bf57 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:49.733149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26437bff-3b05-479a-9bc3-fe1e4fa8bf57 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:49.733625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26437bff-3b05-479a-9bc3-fe1e4fa8bf57 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:49.734053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26437bff-3b05-479a-9bc3-fe1e4fa8bf57 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:49.734734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26437bff-3b05-479a-9bc3-fe1e4fa8bf57 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:49.735192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26437bff-3b05-479a-9bc3-fe1e4fa8bf57 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:49.735612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26437bff-3b05-479a-9bc3-fe1e4fa8bf57 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:50.011853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-26437bff-3b05-479a-9bc3-fe1e4fa8bf57 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "POST /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.292073 Aug 24 10:06:50.012493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 268/550] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:06:49 2022] POST /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments => generated 194 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:50.510317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:50.510922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:50.518033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:50.518513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:50.518941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:50.519622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:50.520244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:50.520679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:50.653685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:50.661486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a160fe31-6fac-4386-9b8f-589fb868c037 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1451 microversion: 2.1 time: 0.155022 Aug 24 10:06:50.662043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 283/551] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:50 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1451 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:50.683570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:50.684316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:50.690928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:50.691415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:50.691853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:50.692595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:50.693049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:50.693475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:50.825500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:50.834153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e927fe0e-18f2-45ca-a0c4-4c21591308aa tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1451 microversion: 2.1 time: 0.155291 Aug 24 10:06:50.834715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 269/552] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:50 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1451 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:52.607180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-b7b768f0-2bfe-4a38-b7e9-8ee2b4a6e9c2 req-2535a89c-ae3a-42c4-a69e-a03e394552a3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "350e1445-9336-439c-b146-ee4c74211b54", "tag": "e7af2444-9e35-49f8-aeab-b94839769f32"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:52.617953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b7b768f0-2bfe-4a38-b7e9-8ee2b4a6e9c2 req-2535a89c-ae3a-42c4-a69e-a03e394552a3 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:52.618441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b7b768f0-2bfe-4a38-b7e9-8ee2b4a6e9c2 req-2535a89c-ae3a-42c4-a69e-a03e394552a3 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:52.618873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b7b768f0-2bfe-4a38-b7e9-8ee2b4a6e9c2 req-2535a89c-ae3a-42c4-a69e-a03e394552a3 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:52.630619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-b7b768f0-2bfe-4a38-b7e9-8ee2b4a6e9c2 req-2535a89c-ae3a-42c4-a69e-a03e394552a3 service nova] Creating event network-changed:e7af2444-9e35-49f8-aeab-b94839769f32 for instance 350e1445-9336-439c-b146-ee4c74211b54 on hv-cinder-85427 Aug 24 10:06:52.635430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-b7b768f0-2bfe-4a38-b7e9-8ee2b4a6e9c2 req-2535a89c-ae3a-42c4-a69e-a03e394552a3 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031893 Aug 24 10:06:52.636026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 284/553] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:53.265539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:53.266166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:53.273736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:53.274229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:53.274668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:53.275369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:53.275821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:53.276269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:53.415988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:53.424151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d5ee4bb8-45c4-474b-a7ec-2dfd47f3b8ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1592 microversion: 2.1 time: 0.163822 Aug 24 10:06:53.424721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 270/554] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:53 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1592 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:55.331360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-7e0e760b-5a98-428b-b51b-fa420464a61d req-7b130200-d3a6-41c5-91e1-f0dc400af6fa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "987654b5-b3b6-434c-a284-d630ab8ab4ad", "tag": "98183aae-275c-45c1-b9fc-a7d038275f2d"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:55.341256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=56 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:55.343488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-7e0e760b-5a98-428b-b51b-fa420464a61d req-7b130200-d3a6-41c5-91e1-f0dc400af6fa service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:55.343986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-7e0e760b-5a98-428b-b51b-fa420464a61d req-7b130200-d3a6-41c5-91e1-f0dc400af6fa service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:55.344490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-7e0e760b-5a98-428b-b51b-fa420464a61d req-7b130200-d3a6-41c5-91e1-f0dc400af6fa service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:55.358160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-7e0e760b-5a98-428b-b51b-fa420464a61d req-7b130200-d3a6-41c5-91e1-f0dc400af6fa service nova] Creating event network-changed:98183aae-275c-45c1-b9fc-a7d038275f2d for instance 987654b5-b3b6-434c-a284-d630ab8ab4ad on hv-cinder-85427 Aug 24 10:06:55.363345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-7e0e760b-5a98-428b-b51b-fa420464a61d req-7b130200-d3a6-41c5-91e1-f0dc400af6fa service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037655 Aug 24 10:06:55.363982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 285/555] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:06:56.415705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1905674680"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:56.416267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:56.422707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:56.423692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:56.424156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:56.425630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:56.425630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:56.425821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:56.568832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.network.security_group_api [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Adding security group cb10e7ee-2c1d-45b7-9519-9ff73d9d547c to port 98183aae-275c-45c1-b9fc-a7d038275f2d Aug 24 10:06:56.924249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-50eb4a68-6460-4652-ab3a-e3913d517b72 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "POST /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/action" status: 202 len: 0 microversion: 2.1 time: 0.511614 Aug 24 10:06:56.924920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 271/556] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:06:56 2022] POST /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/action => generated 0 bytes in 514 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:06:56.934135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-61ef49fe-8459-4807-bc40-d41e738df124 req-842b0a7b-ed44-475e-bb13-e3df28c77e1a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "987654b5-b3b6-434c-a284-d630ab8ab4ad", "tag": "98183aae-275c-45c1-b9fc-a7d038275f2d"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:56.943463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:56.944439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:56.952083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:56.952585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:56.953022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:56.953729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:56.954182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:56.955149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:56.956176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-61ef49fe-8459-4807-bc40-d41e738df124 req-842b0a7b-ed44-475e-bb13-e3df28c77e1a service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:56.956886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-61ef49fe-8459-4807-bc40-d41e738df124 req-842b0a7b-ed44-475e-bb13-e3df28c77e1a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:56.957565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-61ef49fe-8459-4807-bc40-d41e738df124 req-842b0a7b-ed44-475e-bb13-e3df28c77e1a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:56.970967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-61ef49fe-8459-4807-bc40-d41e738df124 req-842b0a7b-ed44-475e-bb13-e3df28c77e1a service nova] Creating event network-changed:98183aae-275c-45c1-b9fc-a7d038275f2d for instance 987654b5-b3b6-434c-a284-d630ab8ab4ad on hv-cinder-85427 Aug 24 10:06:56.978904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-61ef49fe-8459-4807-bc40-d41e738df124 req-842b0a7b-ed44-475e-bb13-e3df28c77e1a service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049393 Aug 24 10:06:56.980393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 286/557] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:06:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:06:57.145098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:06:57.172515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-936387f0-251c-4624-8ec0-1a7420530cf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1757 microversion: 2.1 time: 0.239173 Aug 24 10:06:57.173246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 272/558] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:56 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1757 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:06:57.534385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:57.535089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:57.542153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:57.542918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:57.543576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:57.544560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:57.545237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:57.545895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:57.674686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lazy-loading 'services' on Instance uuid 5692cc07-3f16-47a4-9993-503b451d9951 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:06:57.808760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-54749aba-0db6-4de7-84e0-8a2ca161ea9f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "DELETE /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments/678c737f-2b39-4a8b-b65e-8026cc57f466" status: 202 len: 0 microversion: 2.1 time: 0.278871 Aug 24 10:06:57.809604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 287/559] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:06:57 2022] DELETE /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments/678c737f-2b39-4a8b-b65e-8026cc57f466 => generated 0 bytes in 281 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:06:59.633731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:06:59.636140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:59.642848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:59.643339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:59.643776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:59.644536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:59.644993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:59.645420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:59.668656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'flavor' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:06:59.824478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-73c06159-47be-4c6f-96c2-53acd00a8f1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/action" status: 200 len: 32 microversion: 2.1 time: 0.194729 Aug 24 10:06:59.825080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 273/560] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:06:59 2022] POST /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/action => generated 32 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 24 10:06:59.837126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:06:59.838039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:06:59.845176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:59.845764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:59.846226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:06:59.847111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:06:59.847571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:06:59.848208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:00.064940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:00.073433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2cedd028-6404-4287-bd47-a7d89111d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.241688 Aug 24 10:07:00.074407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 288/561] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:06:59 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:01.096754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:01.097512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:01.105111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:01.105605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:01.106045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:01.106743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:01.107200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:01.107652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:01.237910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:01.246163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ed611c3e-bfd2-429a-9716-8f5551f5ba69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154727 Aug 24 10:07:01.246660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 274/562] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:01 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:02.266745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:02.267381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:02.273798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:02.274281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:02.274852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:02.275613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:02.276092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:02.276526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:02.409772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:02.418326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a3b94ea2-6886-44bb-9c00-1696a46b3db3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.155380 Aug 24 10:07:02.418946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 289/563] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:02 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:03.440381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:03.440995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:03.447342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:03.447821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:03.448296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:03.448987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:03.449444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:03.449871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:03.576136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:03.584722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4df837ba-a370-476e-aeb1-4f4175977d32 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148774 Aug 24 10:07:03.585273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 275/564] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:03 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:04.605978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:04.606612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:04.612967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:04.613463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:04.613901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:04.614606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:04.615064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:04.615489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:04.618256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:04.751579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:04.759759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8ca25b93-981b-4af8-b812-084ec79642d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.157631 Aug 24 10:07:04.760366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 290/565] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:04 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:05.781492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:05.782092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:05.789086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:05.789501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:05.790219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:05.790616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:05.791062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:05.791480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:05.925156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:05.933533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4e1fb2a6-05d4-4167-8cc1-846c0d25d2de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.155869 Aug 24 10:07:05.934090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 276/566] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:05 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:06.954633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:06.955240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:06.961716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:06.962196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:06.962630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:06.963310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:06.963760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:06.964348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:07.095003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:07.103590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-91a73de2-cb53-4249-99e3-125b02061801 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152909 Aug 24 10:07:07.104178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 291/567] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:06 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:08.124093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:08.124638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:08.131058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:08.131536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:08.131968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:08.132669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:08.133117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:08.133541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:08.137551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=6,nova_cell1:INSERT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:08.271429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:08.280417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ec65f37b-67a3-4153-b03d-4fe780e46d6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.159913 Aug 24 10:07:08.280997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 277/568] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:09.302674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:09.303291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:09.309641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:09.310132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:09.310582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:09.311279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:09.311739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:09.312203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:09.444388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:09.452616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b6977c73-3b1c-4511-ae0e-334158f709c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154387 Aug 24 10:07:09.453179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 292/569] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:09 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:10.475197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:10.475793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:10.482423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:10.482900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:10.483328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:10.484002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:10.484472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:10.484885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:10.614221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:10.622457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-37b1cdd5-1d2f-45a3-b677-c014c678801d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151839 Aug 24 10:07:10.623015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 278/570] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:10 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:11.644636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:11.645211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:11.651811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:11.652340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:11.652785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:11.653487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:11.653943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:11.654374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:11.781281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:11.789977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7c374ed1-bec1-434d-a998-4858259a2871 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149267 Aug 24 10:07:11.790538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 293/571] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:11 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:12.811866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:12.812500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:12.814820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=29 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:12.820531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:12.821360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:12.821478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:12.822097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:12.822624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:12.822952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:12.957446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:12.965671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-90070662-6338-4bf2-b97e-dd55dc3dad4d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.157630 Aug 24 10:07:12.966224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 279/572] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:13.988566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:13.989244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:13.995424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:13.995908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:13.996372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:13.997053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:13.997506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:13.997929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:14.129220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:14.138134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fad37ac3-607a-48d5-9246-0eaba859a457 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153518 Aug 24 10:07:14.138710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 294/573] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:13 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:15.159443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:15.159966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:15.167081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:15.167567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:15.168001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:15.168712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:15.169164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:15.169589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:15.295544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:15.303663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-52b91f8e-ee15-4944-8d2e-2c1f4321b6bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148267 Aug 24 10:07:15.304317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 280/574] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:15 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:16.325434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:16.326050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:16.333008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:16.333501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:16.333943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:16.334644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:16.335193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:16.335541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:16.760292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:16.768680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-293b1a02-8d24-4191-8181-a9d97e2f4a45 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.447119 Aug 24 10:07:16.769418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 295/575] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:16 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:17.791443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:17.792074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:17.798524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:17.799009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:17.799448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:17.800173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:17.800628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:17.801634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:17.930156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:17.938402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3d6765d4-a6fa-489b-9d11-8e03660db1ae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150870 Aug 24 10:07:17.938972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 281/576] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:17 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:07:18.958566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:18.959185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:18.965919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:18.966404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:18.966842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:18.967536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:18.967990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:18.968442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:19.099381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:19.107961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e7c20eab-97aa-461c-82c7-ca6fcf91a505 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153361 Aug 24 10:07:19.108549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 296/577] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:18 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:20.129670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:20.130283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:20.136725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:20.137207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:20.137644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:20.138330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:20.138806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:20.139232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:20.266051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:20.274774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1bf8aa73-9fde-4a39-9980-61c15b1928c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149020 Aug 24 10:07:20.275336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 282/578] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:20 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:21.296124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:21.296738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:21.303838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:21.304348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:21.304787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:21.305474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:21.305927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:21.306352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:21.432089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:21.440404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a8d0e8a8-0250-45e6-8858-3dd9d7de5fd7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148077 Aug 24 10:07:21.440960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 297/579] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:21 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:22.461261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:22.461859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:22.468205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:22.468681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:22.469109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:22.469787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:22.470230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:22.470647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:22.598318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:22.606705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-81d4751e-bbb3-4e20-a9de-e26b0033deea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149303 Aug 24 10:07:22.607271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 283/580] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:22 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:22.761729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-55856e94-8f6a-43f6-9c26-4689266fef36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:07:22.763945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-55856e94-8f6a-43f6-9c26-4689266fef36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:22.770469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856e94-8f6a-43f6-9c26-4689266fef36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:22.770917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856e94-8f6a-43f6-9c26-4689266fef36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:22.771353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856e94-8f6a-43f6-9c26-4689266fef36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:22.772132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856e94-8f6a-43f6-9c26-4689266fef36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:22.772528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856e94-8f6a-43f6-9c26-4689266fef36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:22.773024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856e94-8f6a-43f6-9c26-4689266fef36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:22.846811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-55856e94-8f6a-43f6-9c26-4689266fef36 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "POST /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/action" status: 202 len: 0 microversion: 2.1 time: 0.088795 Aug 24 10:07:22.848205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 298/581] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:07:22 2022] POST /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:07:22.856491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:22.857096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:22.864609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:22.865121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:22.865553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:22.866237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:22.866811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:22.867216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:23.020682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:23.029332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-eda1691c-08c1-4e49-9947-44f4f11f20e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1749 microversion: 2.1 time: 0.176623 Aug 24 10:07:23.029991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 284/582] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:22 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1749 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:07:23.629794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:23.630404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:23.636668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:23.637162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:23.637603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:23.638306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:23.638762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:23.639195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:23.779666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:23.788687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-612ee817-abac-4b4b-8d62-3c097f01c272 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.163419 Aug 24 10:07:23.789299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 299/583] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:23 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:24.052227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:24.052834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:24.059108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:24.059596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:24.060032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:24.060735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:24.061217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:24.061640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:24.201887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:24.210601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e6e28bd2-3862-4d43-8aff-088364ccfb84 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.162824 Aug 24 10:07:24.211165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 285/584] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:24 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:24.809390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:24.810009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:24.816511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:24.816997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:24.817437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:24.818125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:24.818586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:24.819018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:24.944711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:24.952923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3aa2d2a1-ae09-40e2-88ba-985176b0417b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.147439 Aug 24 10:07:24.953490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 300/585] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:24 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:25.232180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:25.232741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:25.238983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:25.239455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:25.239884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:25.240622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:25.241067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:25.241480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:25.384791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:25.392926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-933160e6-cc80-464b-acb4-4660d65d8e94 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.164648 Aug 24 10:07:25.393494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 286/586] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:25 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:25.975536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:25.976187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:25.978559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=32 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:25.983821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:25.984349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:25.984794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:25.985491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:25.985948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:25.986375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:26.113832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:26.122455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8c1fdcdb-3b2c-4bab-8e82-82620cf06ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151747 Aug 24 10:07:26.123033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 301/587] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:25 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:26.413955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:26.414622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:26.421055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:26.421533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:26.421968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:26.422652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:26.423103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:26.423526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:26.459366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0e9a0d48-c4b3-4300-b840-213859abdfe5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:26.459873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0e9a0d48-c4b3-4300-b840-213859abdfe5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:26.465926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e9a0d48-c4b3-4300-b840-213859abdfe5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:26.466421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e9a0d48-c4b3-4300-b840-213859abdfe5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:26.466864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e9a0d48-c4b3-4300-b840-213859abdfe5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:26.467569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e9a0d48-c4b3-4300-b840-213859abdfe5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:26.468026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e9a0d48-c4b3-4300-b840-213859abdfe5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:26.468477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e9a0d48-c4b3-4300-b840-213859abdfe5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:26.497775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0e9a0d48-c4b3-4300-b840-213859abdfe5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments/c199d515-fa41-43ae-9e8f-cd84304ee49d" status: 200 len: 194 microversion: 2.20 time: 0.042876 Aug 24 10:07:26.498367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 302/588] 10.222.0.86 () {60 vars in 1454 bytes} [Wed Aug 24 10:07:26 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments/c199d515-fa41-43ae-9e8f-cd84304ee49d => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:07:26.564006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:26.564196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:26.564545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:26.572713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:26.573214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:26.573663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:26.573806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-160e3cbb-7387-4bfd-a267-cdec931c6ad1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.164341 Aug 24 10:07:26.574371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:26.574475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 287/589] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:26 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:26.574836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:26.575268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:26.656837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lazy-loading 'services' on Instance uuid 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:07:26.728085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e86ae114-f9cf-4d75-b121-2f2018ec6dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "DELETE /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments/c199d515-fa41-43ae-9e8f-cd84304ee49d" status: 202 len: 0 microversion: 2.20 time: 0.167898 Aug 24 10:07:26.728659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 303/590] 10.222.0.86 () {60 vars in 1457 bytes} [Wed Aug 24 10:07:26 2022] DELETE /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments/c199d515-fa41-43ae-9e8f-cd84304ee49d => generated 0 bytes in 169 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Aug 24 10:07:26.737117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b25f71a3-c217-4477-ab7d-ab67a9435c56 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:26.739992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b25f71a3-c217-4477-ab7d-ab67a9435c56 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:26.746006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b25f71a3-c217-4477-ab7d-ab67a9435c56 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:26.746493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b25f71a3-c217-4477-ab7d-ab67a9435c56 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:26.746932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b25f71a3-c217-4477-ab7d-ab67a9435c56 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:26.747626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b25f71a3-c217-4477-ab7d-ab67a9435c56 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:26.748108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b25f71a3-c217-4477-ab7d-ab67a9435c56 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:26.748536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b25f71a3-c217-4477-ab7d-ab67a9435c56 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:26.778201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b25f71a3-c217-4477-ab7d-ab67a9435c56 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045127 Aug 24 10:07:26.778761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 288/591] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:26 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:27.143612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:27.144949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:27.152221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:27.152784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:27.153229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:27.153978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:27.154440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:27.154870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:27.333597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:27.342637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4207568d-f607-423a-a67b-97dd9c389e70 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.203608 Aug 24 10:07:27.343195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 304/592] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:27 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:27.594572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:27.595183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:27.601752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:27.602232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:27.602656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:27.603341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:27.603789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:27.604241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:27.741428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:27.749643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-98e39690-1040-4bb1-bc9b-e8b5c96c30e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.158875 Aug 24 10:07:27.750238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 289/593] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:27 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:27.790703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-46d9ae14-dea1-4e44-aac7-2432d0ea0ee1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:27.792956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-46d9ae14-dea1-4e44-aac7-2432d0ea0ee1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:27.798903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-46d9ae14-dea1-4e44-aac7-2432d0ea0ee1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:27.799383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-46d9ae14-dea1-4e44-aac7-2432d0ea0ee1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:27.799825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-46d9ae14-dea1-4e44-aac7-2432d0ea0ee1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:27.801029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-46d9ae14-dea1-4e44-aac7-2432d0ea0ee1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:27.801486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-46d9ae14-dea1-4e44-aac7-2432d0ea0ee1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:27.801912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-46d9ae14-dea1-4e44-aac7-2432d0ea0ee1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:27.804383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:27.836733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-46d9ae14-dea1-4e44-aac7-2432d0ea0ee1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049924 Aug 24 10:07:27.837297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 305/594] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:27 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:28.364945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:28.365551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:28.371507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:28.371986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:28.372518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:28.373206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:28.373653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:28.374075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:28.508677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:28.516973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2050a897-0df1-4238-be5e-ec57e5231711 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.156635 Aug 24 10:07:28.517520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 290/595] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:28 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:28.770901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:28.771286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:28.777625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:28.778113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:28.778556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:28.779247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:28.779714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:28.780175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:28.849318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bfbf3752-96b9-4dbc-825b-740beac9707f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:28.852189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bfbf3752-96b9-4dbc-825b-740beac9707f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:28.859937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfbf3752-96b9-4dbc-825b-740beac9707f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:28.860325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfbf3752-96b9-4dbc-825b-740beac9707f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:28.860751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfbf3752-96b9-4dbc-825b-740beac9707f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:28.861437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfbf3752-96b9-4dbc-825b-740beac9707f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:28.861883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfbf3752-96b9-4dbc-825b-740beac9707f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:28.862381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfbf3752-96b9-4dbc-825b-740beac9707f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:28.893544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bfbf3752-96b9-4dbc-825b-740beac9707f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048614 Aug 24 10:07:28.894027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 291/596] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:28 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:07:28.926663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:28.934830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-856f5cf6-a4e1-475b-a072-008672838822 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.168054 Aug 24 10:07:28.935393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 306/597] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:28 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:29.537927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:29.538562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:29.544478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.544949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.545383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.546124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.546516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.546939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.674596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:29.683211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-20b3edfd-63e2-4f91-973a-738b9b3b1a9e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1716 microversion: 2.1 time: 0.149146 Aug 24 10:07:29.683802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 292/598] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:29 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1716 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:29.791099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:29.791600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:29.798137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.798626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.799067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.799760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.800248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.800679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.879875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Cannot 'detach_volume' instance bdb467be-865b-489c-ab6c-f714a02672e1 while it is in vm_state rescued Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:07:29.881614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:07:29.883479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Cannot 'detach_volume' instance bdb467be-865b-489c-ab6c-f714a02672e1 while it is in vm_state rescued Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance bdb467be-865b-489c-ab6c-f714a02672e1 in vm_state rescued. Cannot detach_volume while the instance is in this state. Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:07:29.884532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance bdb467be-865b-489c-ab6c-f714a02672e1 while it is in vm_state rescued Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance bdb467be-865b-489c-ab6c-f714a02672e1 while it is in vm_state rescued Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:07:29.886183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 409 to user: Cannot 'detach_volume' instance bdb467be-865b-489c-ab6c-f714a02672e1 while it is in vm_state rescued {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:07:29.887836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ec1294eb-626f-4ff5-852e-d7603b42a112 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 409 len: 152 microversion: 2.1 time: 0.095945 Aug 24 10:07:29.887836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 307/599] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:07:29 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 152 bytes in 97 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Aug 24 10:07:29.892031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-929b0861-09dd-4421-99f3-dace6a50e38b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:07:29.892583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-929b0861-09dd-4421-99f3-dace6a50e38b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:29.899272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-929b0861-09dd-4421-99f3-dace6a50e38b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.899862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-929b0861-09dd-4421-99f3-dace6a50e38b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.900207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-929b0861-09dd-4421-99f3-dace6a50e38b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.900903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-929b0861-09dd-4421-99f3-dace6a50e38b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.901375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-929b0861-09dd-4421-99f3-dace6a50e38b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.901819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-929b0861-09dd-4421-99f3-dace6a50e38b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.910623 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-913555a6-993b-48cc-90d6-db4142898bf8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:29.912816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-913555a6-993b-48cc-90d6-db4142898bf8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:29.919554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913555a6-993b-48cc-90d6-db4142898bf8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.920037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913555a6-993b-48cc-90d6-db4142898bf8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.920493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913555a6-993b-48cc-90d6-db4142898bf8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.921178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913555a6-993b-48cc-90d6-db4142898bf8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.921638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913555a6-993b-48cc-90d6-db4142898bf8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.922065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913555a6-993b-48cc-90d6-db4142898bf8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.929444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:29.952161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-913555a6-993b-48cc-90d6-db4142898bf8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045298 Aug 24 10:07:29.952726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 308/600] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:07:29.957263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:29.957871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:29.963640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.964158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.964602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.965303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.965743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.966168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.971504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-929b0861-09dd-4421-99f3-dace6a50e38b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/action" status: 202 len: 0 microversion: 2.1 time: 0.083017 Aug 24 10:07:29.972728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 293/601] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:07:29 2022] POST /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/action => generated 0 bytes in 85 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:07:29.981309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:29.982723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:29.989889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.990455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.990994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:29.991823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:29.992364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:29.992800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:30.130568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:30.139005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3ea8b12e-8d49-4d04-9b67-6bc3a77f9244 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1724 microversion: 2.1 time: 0.161743 Aug 24 10:07:30.139578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 294/602] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:29 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1724 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:07:30.214200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:30.222844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e21cd290-93bd-465f-9bc9-95d3c1c77367 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.269102 Aug 24 10:07:30.223407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 309/603] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:29 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:30.966156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a5af4f1b-a8b8-4753-ab7d-8998cff66627 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:30.968381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a5af4f1b-a8b8-4753-ab7d-8998cff66627 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:30.975544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5af4f1b-a8b8-4753-ab7d-8998cff66627 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:30.976023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5af4f1b-a8b8-4753-ab7d-8998cff66627 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:30.976480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5af4f1b-a8b8-4753-ab7d-8998cff66627 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:30.977168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5af4f1b-a8b8-4753-ab7d-8998cff66627 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:30.977620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5af4f1b-a8b8-4753-ab7d-8998cff66627 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:30.978050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5af4f1b-a8b8-4753-ab7d-8998cff66627 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:31.009710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a5af4f1b-a8b8-4753-ab7d-8998cff66627 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048415 Aug 24 10:07:31.011110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 295/604] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:31.160247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:31.160910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:31.167306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:31.167796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:31.168275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:31.168972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:31.169437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:31.169865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:31.242707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:31.243303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:31.249179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:31.249655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:31.250083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:31.250759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:31.251201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:31.251616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:31.305190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:31.313928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-22d9b0d1-988e-4931-92d7-efd608c42667 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1724 microversion: 2.1 time: 0.157573 Aug 24 10:07:31.314775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 310/605] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:31 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1724 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:31.396280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:31.404521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2fb83838-de30-4126-aea3-ae3894f61b1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.165653 Aug 24 10:07:31.405080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 296/606] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:31 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:07:32.022948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9e5c55df-13d7-433f-bd3a-50db332dc1c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:32.025174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9e5c55df-13d7-433f-bd3a-50db332dc1c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:32.031066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e5c55df-13d7-433f-bd3a-50db332dc1c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:32.031546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e5c55df-13d7-433f-bd3a-50db332dc1c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:32.031976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e5c55df-13d7-433f-bd3a-50db332dc1c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:32.032678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e5c55df-13d7-433f-bd3a-50db332dc1c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:32.033150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e5c55df-13d7-433f-bd3a-50db332dc1c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:32.033570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e5c55df-13d7-433f-bd3a-50db332dc1c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:32.062626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9e5c55df-13d7-433f-bd3a-50db332dc1c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043690 Aug 24 10:07:32.063162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 311/607] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:32 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:32.335652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:32.336317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:32.343451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:32.343923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:32.344391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:32.345071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:32.345518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:32.345933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:32.425838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:32.426450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:32.433147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:32.433646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:32.434088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:32.434791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:32.435247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:32.435679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:32.477969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:32.486228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fdf48769-2224-4cd3-9dce-2d2095d66944 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1724 microversion: 2.1 time: 0.154502 Aug 24 10:07:32.486801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 297/608] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:32 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1724 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:32.578394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:32.588008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4cacb2ca-ddda-48a3-8520-61c5044685e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.166036 Aug 24 10:07:32.588553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 312/609] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:32 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:07:33.074686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-802d70e2-4b93-4892-8aba-0e5f7afc1c73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:33.076889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-802d70e2-4b93-4892-8aba-0e5f7afc1c73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:33.083367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-802d70e2-4b93-4892-8aba-0e5f7afc1c73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:33.083845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-802d70e2-4b93-4892-8aba-0e5f7afc1c73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:33.084307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-802d70e2-4b93-4892-8aba-0e5f7afc1c73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:33.084985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-802d70e2-4b93-4892-8aba-0e5f7afc1c73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:33.085431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-802d70e2-4b93-4892-8aba-0e5f7afc1c73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:33.085847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-802d70e2-4b93-4892-8aba-0e5f7afc1c73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:33.115081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-802d70e2-4b93-4892-8aba-0e5f7afc1c73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044260 Aug 24 10:07:33.115844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 298/610] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:33.507983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:33.508650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:33.515003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:33.515485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:33.515923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:33.516637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:33.517100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:33.517526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:33.608798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:33.609412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:33.616454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:33.616945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:33.617381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:33.618074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:33.618526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:33.618948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:33.649285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:33.659143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1e3c5ed4-d135-4ba4-9897-50dd8eea9d8f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1730 microversion: 2.1 time: 0.154998 Aug 24 10:07:33.659880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 313/611] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:33 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1730 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:33.680223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-418aca32-ce4f-4302-8bf4-b38907e58ba2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:33.680718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-418aca32-ce4f-4302-8bf4-b38907e58ba2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:33.686689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-418aca32-ce4f-4302-8bf4-b38907e58ba2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:33.687178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-418aca32-ce4f-4302-8bf4-b38907e58ba2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:33.687615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-418aca32-ce4f-4302-8bf4-b38907e58ba2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:33.688334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-418aca32-ce4f-4302-8bf4-b38907e58ba2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:33.688793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-418aca32-ce4f-4302-8bf4-b38907e58ba2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:33.689217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-418aca32-ce4f-4302-8bf4-b38907e58ba2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:33.765389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:33.775555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-61cf8dc7-8f40-47bc-be7d-3c6afb1552d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.170439 Aug 24 10:07:33.782455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-418aca32-ce4f-4302-8bf4-b38907e58ba2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.106065 Aug 24 10:07:33.789607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 299/612] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:33 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:33.791063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 314/613] 10.222.0.86 () {58 vars in 1275 bytes} [Wed Aug 24 10:07:33 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-interface => generated 284 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:07:34.127353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1fa709d6-ec89-4302-af1e-6cfdc31d7795 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:34.129884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1fa709d6-ec89-4302-af1e-6cfdc31d7795 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:34.135882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa709d6-ec89-4302-af1e-6cfdc31d7795 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:34.136406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa709d6-ec89-4302-af1e-6cfdc31d7795 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:34.136842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa709d6-ec89-4302-af1e-6cfdc31d7795 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:34.137539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa709d6-ec89-4302-af1e-6cfdc31d7795 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:34.137991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa709d6-ec89-4302-af1e-6cfdc31d7795 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:34.138411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa709d6-ec89-4302-af1e-6cfdc31d7795 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:34.168412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1fa709d6-ec89-4302-af1e-6cfdc31d7795 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045133 Aug 24 10:07:34.169198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 300/614] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:34 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:34.684737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-7ab938ab-7459-4dbb-8d90-9dec16a39db1 req-ce6b1ea3-9e0f-4fa3-acef-0007b8d322e3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bdb467be-865b-489c-ab6c-f714a02672e1", "tag": "7b852e03-cec0-4436-b50c-9c652193755e"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:07:34.696089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-7ab938ab-7459-4dbb-8d90-9dec16a39db1 req-ce6b1ea3-9e0f-4fa3-acef-0007b8d322e3 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:34.696848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-7ab938ab-7459-4dbb-8d90-9dec16a39db1 req-ce6b1ea3-9e0f-4fa3-acef-0007b8d322e3 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:34.697495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-7ab938ab-7459-4dbb-8d90-9dec16a39db1 req-ce6b1ea3-9e0f-4fa3-acef-0007b8d322e3 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:34.712307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-7ab938ab-7459-4dbb-8d90-9dec16a39db1 req-ce6b1ea3-9e0f-4fa3-acef-0007b8d322e3 service nova] Creating event network-changed:7b852e03-cec0-4436-b50c-9c652193755e for instance bdb467be-865b-489c-ab6c-f714a02672e1 on hv-cinder-85427 Aug 24 10:07:34.718582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-7ab938ab-7459-4dbb-8d90-9dec16a39db1 req-ce6b1ea3-9e0f-4fa3-acef-0007b8d322e3 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037583 Aug 24 10:07:34.719315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 315/615] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:07:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:07:34.796930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:34.797627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:34.804414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:34.805112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:34.805616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:34.806571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:34.807186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:34.809091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:34.974331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:34.983034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1d83edb1-ca1f-4aeb-b6a0-95434374ec14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.190180 Aug 24 10:07:34.997165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 301/616] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:34 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:35.182439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-714287a7-1a15-4d02-8547-2fb4d1b71b8d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:35.185553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-714287a7-1a15-4d02-8547-2fb4d1b71b8d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:35.192825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-714287a7-1a15-4d02-8547-2fb4d1b71b8d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:35.193318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-714287a7-1a15-4d02-8547-2fb4d1b71b8d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:35.193751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-714287a7-1a15-4d02-8547-2fb4d1b71b8d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:35.194471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-714287a7-1a15-4d02-8547-2fb4d1b71b8d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:35.194923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-714287a7-1a15-4d02-8547-2fb4d1b71b8d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:35.195347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-714287a7-1a15-4d02-8547-2fb4d1b71b8d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:35.226849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-714287a7-1a15-4d02-8547-2fb4d1b71b8d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049276 Aug 24 10:07:35.227407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 316/617] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:35 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:36.004203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:36.004808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:36.011147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:36.011615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:36.012092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:36.012769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:36.013246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:36.013665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:36.161671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:36.169861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4b72888c-ab2e-4559-bd33-274056784211 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.169554 Aug 24 10:07:36.170425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 302/618] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:35 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:36.238833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b74dd4de-beda-4a28-88b0-2ad4e5f279ca tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:36.240977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b74dd4de-beda-4a28-88b0-2ad4e5f279ca tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:36.247281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b74dd4de-beda-4a28-88b0-2ad4e5f279ca tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:36.247772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b74dd4de-beda-4a28-88b0-2ad4e5f279ca tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:36.248258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b74dd4de-beda-4a28-88b0-2ad4e5f279ca tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:36.248951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b74dd4de-beda-4a28-88b0-2ad4e5f279ca tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:36.249413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b74dd4de-beda-4a28-88b0-2ad4e5f279ca tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:36.249839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b74dd4de-beda-4a28-88b0-2ad4e5f279ca tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:36.279239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b74dd4de-beda-4a28-88b0-2ad4e5f279ca tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044364 Aug 24 10:07:36.279795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 317/619] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:36 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:36.735093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-15987ba1-d9ee-40c1-bead-7ba36cb43ffb req-61148241-9572-495c-b3ea-b6f220982cd1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bdb467be-865b-489c-ab6c-f714a02672e1", "tag": "7b852e03-cec0-4436-b50c-9c652193755e"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:07:36.744707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-15987ba1-d9ee-40c1-bead-7ba36cb43ffb req-61148241-9572-495c-b3ea-b6f220982cd1 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:36.745163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-15987ba1-d9ee-40c1-bead-7ba36cb43ffb req-61148241-9572-495c-b3ea-b6f220982cd1 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:36.745604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-15987ba1-d9ee-40c1-bead-7ba36cb43ffb req-61148241-9572-495c-b3ea-b6f220982cd1 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:36.757082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-15987ba1-d9ee-40c1-bead-7ba36cb43ffb req-61148241-9572-495c-b3ea-b6f220982cd1 service nova] Creating event network-changed:7b852e03-cec0-4436-b50c-9c652193755e for instance bdb467be-865b-489c-ab6c-f714a02672e1 on hv-cinder-85427 Aug 24 10:07:36.762144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-15987ba1-d9ee-40c1-bead-7ba36cb43ffb req-61148241-9572-495c-b3ea-b6f220982cd1 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030629 Aug 24 10:07:36.762821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 303/620] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:07:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:07:37.190714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:37.191336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:37.197411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:37.197907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:37.198351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:37.199059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:37.199520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:37.199953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:37.292182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e601eefa-699a-4dc5-84a4-27ad2a710677 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:37.294315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e601eefa-699a-4dc5-84a4-27ad2a710677 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:37.301690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e601eefa-699a-4dc5-84a4-27ad2a710677 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:37.302171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e601eefa-699a-4dc5-84a4-27ad2a710677 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:37.302599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e601eefa-699a-4dc5-84a4-27ad2a710677 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:37.303297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e601eefa-699a-4dc5-84a4-27ad2a710677 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:37.303742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e601eefa-699a-4dc5-84a4-27ad2a710677 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:37.304865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e601eefa-699a-4dc5-84a4-27ad2a710677 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:37.334271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e601eefa-699a-4dc5-84a4-27ad2a710677 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046545 Aug 24 10:07:37.335035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 304/621] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:07:37.389661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:37.398327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-52fadb63-a97f-4cdf-b66c-4d84fe1c05e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.211441 Aug 24 10:07:37.398895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 318/622] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:37 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:07:38.346860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3f8c7e04-94d7-4d27-9fb6-1c8da8b8f72b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:38.348936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3f8c7e04-94d7-4d27-9fb6-1c8da8b8f72b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:38.355442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f8c7e04-94d7-4d27-9fb6-1c8da8b8f72b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:38.355917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f8c7e04-94d7-4d27-9fb6-1c8da8b8f72b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:38.356447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f8c7e04-94d7-4d27-9fb6-1c8da8b8f72b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:38.357131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f8c7e04-94d7-4d27-9fb6-1c8da8b8f72b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:38.357580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f8c7e04-94d7-4d27-9fb6-1c8da8b8f72b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:38.357996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f8c7e04-94d7-4d27-9fb6-1c8da8b8f72b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:38.387191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3f8c7e04-94d7-4d27-9fb6-1c8da8b8f72b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044375 Aug 24 10:07:38.387728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 305/623] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:38 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:38.419723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:38.420407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:38.427339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:38.427840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:38.428301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:38.428992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:38.429471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:38.429896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:38.569385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:38.578022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-639b8927-7354-4b1d-8392-992531174bf2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.162181 Aug 24 10:07:38.578579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 319/624] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:38 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:39.402876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7521b10e-2cfb-49a1-ac15-6a3cf1ec2969 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:39.405377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7521b10e-2cfb-49a1-ac15-6a3cf1ec2969 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:39.411825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7521b10e-2cfb-49a1-ac15-6a3cf1ec2969 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:39.412407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7521b10e-2cfb-49a1-ac15-6a3cf1ec2969 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:39.412847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7521b10e-2cfb-49a1-ac15-6a3cf1ec2969 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:39.413547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7521b10e-2cfb-49a1-ac15-6a3cf1ec2969 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:39.414006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7521b10e-2cfb-49a1-ac15-6a3cf1ec2969 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:39.414434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7521b10e-2cfb-49a1-ac15-6a3cf1ec2969 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:39.444417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7521b10e-2cfb-49a1-ac15-6a3cf1ec2969 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047974 Aug 24 10:07:39.444984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 306/625] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:39 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:39.598650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:39.599256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:39.606430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:39.606931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:39.607363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:39.608059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:39.608516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:39.608935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:39.754467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:39.762831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f6301bd3-e87a-42dd-8add-6ed848b1df55 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1754 microversion: 2.1 time: 0.168017 Aug 24 10:07:39.763400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 320/626] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:39 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1754 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:40.459752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2e3d13e3-dc39-458f-8d7f-11282cbb8ac1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:40.461928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2e3d13e3-dc39-458f-8d7f-11282cbb8ac1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:40.468401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e3d13e3-dc39-458f-8d7f-11282cbb8ac1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:40.468881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e3d13e3-dc39-458f-8d7f-11282cbb8ac1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:40.469313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e3d13e3-dc39-458f-8d7f-11282cbb8ac1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:40.470021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e3d13e3-dc39-458f-8d7f-11282cbb8ac1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:40.470472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e3d13e3-dc39-458f-8d7f-11282cbb8ac1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:40.470891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e3d13e3-dc39-458f-8d7f-11282cbb8ac1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:40.500198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2e3d13e3-dc39-458f-8d7f-11282cbb8ac1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044428 Aug 24 10:07:40.500753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 307/627] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:40.782657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:40.783268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:40.790596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:40.791088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:40.791528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:40.792274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:40.792736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:40.793167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:40.931343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:40.939489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-df996e99-f25e-4994-aa70-035164daa7b5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1757 microversion: 2.1 time: 0.160646 Aug 24 10:07:40.940060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 321/628] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:40 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1757 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:41.513256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ed56a002-65bf-4d68-aaac-2f2390ae098b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:41.515372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ed56a002-65bf-4d68-aaac-2f2390ae098b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:41.522739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed56a002-65bf-4d68-aaac-2f2390ae098b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:41.523222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed56a002-65bf-4d68-aaac-2f2390ae098b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:41.523768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed56a002-65bf-4d68-aaac-2f2390ae098b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:41.524497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed56a002-65bf-4d68-aaac-2f2390ae098b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:41.524949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed56a002-65bf-4d68-aaac-2f2390ae098b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:41.525394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed56a002-65bf-4d68-aaac-2f2390ae098b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:41.553725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ed56a002-65bf-4d68-aaac-2f2390ae098b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044513 Aug 24 10:07:41.554271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 308/629] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:41 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:42.566346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-26ecea98-b12c-41ef-97d9-b24f989c086a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:42.568497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-26ecea98-b12c-41ef-97d9-b24f989c086a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:42.574922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26ecea98-b12c-41ef-97d9-b24f989c086a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:42.575408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26ecea98-b12c-41ef-97d9-b24f989c086a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:42.575850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26ecea98-b12c-41ef-97d9-b24f989c086a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:42.576643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26ecea98-b12c-41ef-97d9-b24f989c086a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:42.577150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26ecea98-b12c-41ef-97d9-b24f989c086a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:42.577612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26ecea98-b12c-41ef-97d9-b24f989c086a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:42.603864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:42.609882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-26ecea98-b12c-41ef-97d9-b24f989c086a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047676 Aug 24 10:07:42.610438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 322/630] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:42 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:43.622518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6d115a34-2746-4506-9e61-3946bb968597 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:43.624592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6d115a34-2746-4506-9e61-3946bb968597 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:43.627329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=60 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:43.633510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d115a34-2746-4506-9e61-3946bb968597 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:43.634028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d115a34-2746-4506-9e61-3946bb968597 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:43.634466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d115a34-2746-4506-9e61-3946bb968597 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:43.635161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d115a34-2746-4506-9e61-3946bb968597 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:43.635616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d115a34-2746-4506-9e61-3946bb968597 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:43.636062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d115a34-2746-4506-9e61-3946bb968597 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:43.664778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6d115a34-2746-4506-9e61-3946bb968597 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046312 Aug 24 10:07:43.665345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 309/631] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:44.678110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bbcfc5d1-881b-45c4-925a-a46172ccc459 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:44.680267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bbcfc5d1-881b-45c4-925a-a46172ccc459 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:44.686713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bbcfc5d1-881b-45c4-925a-a46172ccc459 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:44.687195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bbcfc5d1-881b-45c4-925a-a46172ccc459 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:44.687659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bbcfc5d1-881b-45c4-925a-a46172ccc459 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:44.688379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bbcfc5d1-881b-45c4-925a-a46172ccc459 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:44.688836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bbcfc5d1-881b-45c4-925a-a46172ccc459 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:44.689255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bbcfc5d1-881b-45c4-925a-a46172ccc459 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:44.721749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bbcfc5d1-881b-45c4-925a-a46172ccc459 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048523 Aug 24 10:07:44.722310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 323/632] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:45.734384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-85bb6373-a9e9-48a7-89af-537813bf34f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:45.736551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-85bb6373-a9e9-48a7-89af-537813bf34f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:45.743167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-85bb6373-a9e9-48a7-89af-537813bf34f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:45.743653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-85bb6373-a9e9-48a7-89af-537813bf34f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:45.744119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-85bb6373-a9e9-48a7-89af-537813bf34f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:45.744819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-85bb6373-a9e9-48a7-89af-537813bf34f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:45.745273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-85bb6373-a9e9-48a7-89af-537813bf34f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:45.745698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-85bb6373-a9e9-48a7-89af-537813bf34f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:45.749347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:45.778004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-85bb6373-a9e9-48a7-89af-537813bf34f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047568 Aug 24 10:07:45.778585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 310/633] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:45 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:46.791002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5f3ee407-91c7-4454-97a5-9ab02150864d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:46.793178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5f3ee407-91c7-4454-97a5-9ab02150864d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:46.799649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3ee407-91c7-4454-97a5-9ab02150864d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:46.800172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3ee407-91c7-4454-97a5-9ab02150864d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:46.800617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3ee407-91c7-4454-97a5-9ab02150864d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:46.802001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3ee407-91c7-4454-97a5-9ab02150864d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:46.802462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3ee407-91c7-4454-97a5-9ab02150864d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:46.802897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3ee407-91c7-4454-97a5-9ab02150864d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:46.831813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5f3ee407-91c7-4454-97a5-9ab02150864d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044826 Aug 24 10:07:46.832403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 324/634] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:46 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:47.844326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b8bfab18-eb9e-40f8-9d93-fde9241bef7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:47.846436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b8bfab18-eb9e-40f8-9d93-fde9241bef7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:47.853140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8bfab18-eb9e-40f8-9d93-fde9241bef7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:47.853621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8bfab18-eb9e-40f8-9d93-fde9241bef7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:47.854066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8bfab18-eb9e-40f8-9d93-fde9241bef7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:47.854742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8bfab18-eb9e-40f8-9d93-fde9241bef7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:47.855195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8bfab18-eb9e-40f8-9d93-fde9241bef7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:47.855622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8bfab18-eb9e-40f8-9d93-fde9241bef7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:47.884855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b8bfab18-eb9e-40f8-9d93-fde9241bef7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044549 Aug 24 10:07:47.885404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 311/635] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:48.034921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:07:48.035468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:48.042025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:48.042518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:48.042953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:48.043643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:48.044126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:48.044556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:48.068189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lazy-loading 'flavor' on Instance uuid 0c601f9d-f26b-4095-a00c-53c422163d31 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:07:48.139319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-99fa8b32-1151-4fa8-b5ee-c1286354997b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "POST /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31/action" status: 202 len: 0 microversion: 2.1 time: 0.108195 Aug 24 10:07:48.139906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 325/636] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:07:48 2022] POST /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31/action => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:07:48.148777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:48.149524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:48.155339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:48.155901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:48.156460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:48.157307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:48.157778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:48.158240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:48.291944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:48.299723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cedfc364-3a21-4851-9333-fa2111fc4fff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.155399 Aug 24 10:07:48.300312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 312/637] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:48 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:48.898345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-72f151a3-5697-44e5-8856-c923bcf39701 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:48.900501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-72f151a3-5697-44e5-8856-c923bcf39701 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:48.906560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72f151a3-5697-44e5-8856-c923bcf39701 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:48.907040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72f151a3-5697-44e5-8856-c923bcf39701 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:48.907490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72f151a3-5697-44e5-8856-c923bcf39701 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:48.908179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72f151a3-5697-44e5-8856-c923bcf39701 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:48.908631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72f151a3-5697-44e5-8856-c923bcf39701 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:48.909055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72f151a3-5697-44e5-8856-c923bcf39701 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:48.937985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-72f151a3-5697-44e5-8856-c923bcf39701 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044633 Aug 24 10:07:48.938525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 326/638] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:48 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:49.376930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:49.377533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:49.384108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:49.384614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:49.385043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:49.385718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:49.386162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:49.386579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:49.520009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:49.528137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fed9649b-d061-4e80-80ec-5c7f71911a95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.155110 Aug 24 10:07:49.528704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 313/639] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:49 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:49.950184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-50913363-59ea-4898-a999-436ab083517b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:49.952372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-50913363-59ea-4898-a999-436ab083517b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:49.958874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50913363-59ea-4898-a999-436ab083517b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:49.959364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50913363-59ea-4898-a999-436ab083517b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:49.959809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50913363-59ea-4898-a999-436ab083517b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:49.960545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50913363-59ea-4898-a999-436ab083517b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:49.961002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50913363-59ea-4898-a999-436ab083517b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:49.961431 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50913363-59ea-4898-a999-436ab083517b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:49.992937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-50913363-59ea-4898-a999-436ab083517b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046714 Aug 24 10:07:49.993502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 327/640] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:50.549189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:50.549805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:50.556500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:50.556980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:50.557412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:50.558129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:50.558583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:50.559007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:50.687154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:50.695548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9c06cd87-6734-4dd1-b200-05861442f128 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.150180 Aug 24 10:07:50.696588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 314/641] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:50 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:51.006029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9ab30d9f-9d31-418e-830f-be2d6b889a49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:51.008148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9ab30d9f-9d31-418e-830f-be2d6b889a49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:51.014610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ab30d9f-9d31-418e-830f-be2d6b889a49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:51.015092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ab30d9f-9d31-418e-830f-be2d6b889a49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:51.015527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ab30d9f-9d31-418e-830f-be2d6b889a49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:51.016252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ab30d9f-9d31-418e-830f-be2d6b889a49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:51.016708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ab30d9f-9d31-418e-830f-be2d6b889a49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:51.017137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ab30d9f-9d31-418e-830f-be2d6b889a49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:51.046078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9ab30d9f-9d31-418e-830f-be2d6b889a49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044780 Aug 24 10:07:51.046647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 328/642] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:51.716526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:51.717145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:51.724196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:51.724685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:51.725127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:51.726085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:51.726538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:51.726962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:51.860368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:51.868107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-04438fdb-464a-4dce-a860-61ac2dad1cf3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.155439 Aug 24 10:07:51.868672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 315/643] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:51 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:52.058650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a62443cb-48a3-4522-91ab-41cc720f54c8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:52.060740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a62443cb-48a3-4522-91ab-41cc720f54c8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:52.067577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a62443cb-48a3-4522-91ab-41cc720f54c8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:52.068079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a62443cb-48a3-4522-91ab-41cc720f54c8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:52.068522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a62443cb-48a3-4522-91ab-41cc720f54c8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:52.069501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a62443cb-48a3-4522-91ab-41cc720f54c8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:52.069664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a62443cb-48a3-4522-91ab-41cc720f54c8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:52.070092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a62443cb-48a3-4522-91ab-41cc720f54c8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:52.099023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a62443cb-48a3-4522-91ab-41cc720f54c8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044496 Aug 24 10:07:52.099567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 329/644] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:52 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:52.891122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:52.891763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:52.898082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:52.898557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:52.898982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:52.899652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:52.900126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:52.900549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:53.031865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:53.040549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-821c8e33-db0b-4006-be83-857f0073e0bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.153112 Aug 24 10:07:53.041126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 316/645] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:52 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:53.111817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a4ba7387-6b1b-4f68-b2f8-23bf49412f57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:53.114668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a4ba7387-6b1b-4f68-b2f8-23bf49412f57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:53.121745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4ba7387-6b1b-4f68-b2f8-23bf49412f57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:53.122564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4ba7387-6b1b-4f68-b2f8-23bf49412f57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:53.123048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4ba7387-6b1b-4f68-b2f8-23bf49412f57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:53.124008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4ba7387-6b1b-4f68-b2f8-23bf49412f57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:53.124521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4ba7387-6b1b-4f68-b2f8-23bf49412f57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:53.125036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4ba7387-6b1b-4f68-b2f8-23bf49412f57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:53.154923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a4ba7387-6b1b-4f68-b2f8-23bf49412f57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047302 Aug 24 10:07:53.155481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 330/646] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:53 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:54.062240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:54.062854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:54.070120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:54.070603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:54.071041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:54.071727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:54.072211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:54.072636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:54.166836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a851a3e1-7118-4cf5-9e35-c9ea53ba8acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:54.169090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a851a3e1-7118-4cf5-9e35-c9ea53ba8acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:54.175293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a851a3e1-7118-4cf5-9e35-c9ea53ba8acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:54.175782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a851a3e1-7118-4cf5-9e35-c9ea53ba8acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:54.176763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a851a3e1-7118-4cf5-9e35-c9ea53ba8acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:54.177786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a851a3e1-7118-4cf5-9e35-c9ea53ba8acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:54.178247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a851a3e1-7118-4cf5-9e35-c9ea53ba8acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:54.178776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a851a3e1-7118-4cf5-9e35-c9ea53ba8acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:54.202729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:54.208749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a851a3e1-7118-4cf5-9e35-c9ea53ba8acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045808 Aug 24 10:07:54.209326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 331/647] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:54 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:07:54.210673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-90118da5-9bc2-4233-a8bc-5c4ee172ba95 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.152269 Aug 24 10:07:54.211235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 317/648] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:54 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:55.221199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-53aa8f01-1cfd-432b-aeb9-959985a2cb80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:55.223331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-53aa8f01-1cfd-432b-aeb9-959985a2cb80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:55.230035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53aa8f01-1cfd-432b-aeb9-959985a2cb80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:55.230530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53aa8f01-1cfd-432b-aeb9-959985a2cb80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:55.230973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53aa8f01-1cfd-432b-aeb9-959985a2cb80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:55.231689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53aa8f01-1cfd-432b-aeb9-959985a2cb80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:55.232208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53aa8f01-1cfd-432b-aeb9-959985a2cb80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:55.232643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53aa8f01-1cfd-432b-aeb9-959985a2cb80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:55.234317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:55.234940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:55.241616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:55.242055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:55.242279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:55.242976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:55.243432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:55.243855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:55.263784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-53aa8f01-1cfd-432b-aeb9-959985a2cb80 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046487 Aug 24 10:07:55.264381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 332/649] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:55 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:07:55.375995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:55.384095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0ed9623c-6a3d-41c9-9b7d-214a247f3b63 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.154956 Aug 24 10:07:55.384655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 318/650] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:55 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:07:56.276882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cd8c7704-9a3f-4279-845a-89633bd60b64 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:56.279018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cd8c7704-9a3f-4279-845a-89633bd60b64 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:56.281432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=64 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:56.287255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8c7704-9a3f-4279-845a-89633bd60b64 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:56.287732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8c7704-9a3f-4279-845a-89633bd60b64 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:56.288202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8c7704-9a3f-4279-845a-89633bd60b64 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:56.288901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8c7704-9a3f-4279-845a-89633bd60b64 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:56.289357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8c7704-9a3f-4279-845a-89633bd60b64 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:56.289783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8c7704-9a3f-4279-845a-89633bd60b64 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:56.318257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cd8c7704-9a3f-4279-845a-89633bd60b64 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045509 Aug 24 10:07:56.318802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 333/651] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:56.405011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:56.405614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:56.412594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:56.413069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:56.413498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:56.414174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:56.414623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:56.415038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:56.541023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:56.548625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b8b4f8b8-5570-4e3e-9410-e097ef965a20 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.147545 Aug 24 10:07:56.549176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 319/652] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:56 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:57.330497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-540a5539-b203-4133-b047-3f5c187000ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:57.332559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-540a5539-b203-4133-b047-3f5c187000ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:57.339200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-540a5539-b203-4133-b047-3f5c187000ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:57.339887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-540a5539-b203-4133-b047-3f5c187000ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:57.340471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-540a5539-b203-4133-b047-3f5c187000ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:57.341244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-540a5539-b203-4133-b047-3f5c187000ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:57.341731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-540a5539-b203-4133-b047-3f5c187000ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:57.342161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-540a5539-b203-4133-b047-3f5c187000ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:57.371701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-540a5539-b203-4133-b047-3f5c187000ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045348 Aug 24 10:07:57.372288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 334/653] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:57 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:57.569543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:57.570148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:57.576541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:57.577029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:57.577460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:57.578149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:57.578602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:57.579026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:57.722209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:57.730847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8d88aca5-88fd-46fd-881c-0b64163e19bc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.165186 Aug 24 10:07:57.731412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 320/654] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:57 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:58.384793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a4f7f815-c835-45be-9229-473443537c75 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:58.386815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a4f7f815-c835-45be-9229-473443537c75 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:58.393099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4f7f815-c835-45be-9229-473443537c75 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:58.393587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4f7f815-c835-45be-9229-473443537c75 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:58.394018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4f7f815-c835-45be-9229-473443537c75 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:58.394705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4f7f815-c835-45be-9229-473443537c75 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:58.395152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4f7f815-c835-45be-9229-473443537c75 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:58.395574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4f7f815-c835-45be-9229-473443537c75 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:58.425721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a4f7f815-c835-45be-9229-473443537c75 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044933 Aug 24 10:07:58.427078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 335/655] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:58 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:07:58.751990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:58.752659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:58.760622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:58.760622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:58.760622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:58.761038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:58.761118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:58.761604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:59.022472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:07:59.030223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e8815167-cd53-48cd-adee-fbfcc72bd92c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.282134 Aug 24 10:07:59.030801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 321/656] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:07:58 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:07:59.438914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-edece3cb-6b8d-40c9-9462-5f02d35c26d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:07:59.441220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-edece3cb-6b8d-40c9-9462-5f02d35c26d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:07:59.448461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edece3cb-6b8d-40c9-9462-5f02d35c26d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:59.448949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edece3cb-6b8d-40c9-9462-5f02d35c26d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:59.449415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edece3cb-6b8d-40c9-9462-5f02d35c26d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:59.450121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edece3cb-6b8d-40c9-9462-5f02d35c26d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:07:59.450740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edece3cb-6b8d-40c9-9462-5f02d35c26d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:07:59.450998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edece3cb-6b8d-40c9-9462-5f02d35c26d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:07:59.482026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-edece3cb-6b8d-40c9-9462-5f02d35c26d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047085 Aug 24 10:07:59.482595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 336/657] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:07:59 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:00.052658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:00.053273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:00.059624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:00.060144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:00.060586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:00.061274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:00.061729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:00.062156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:00.190391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:00.198842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-400fe224-f69b-42fc-a0d9-803e358f6b82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.150502 Aug 24 10:08:00.202256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 322/658] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:00 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:00.496829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-75269c10-feda-42c4-bb45-66db5bdb734c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:00.498923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-75269c10-feda-42c4-bb45-66db5bdb734c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:00.505350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75269c10-feda-42c4-bb45-66db5bdb734c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:00.505828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75269c10-feda-42c4-bb45-66db5bdb734c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:00.506261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75269c10-feda-42c4-bb45-66db5bdb734c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:00.506953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75269c10-feda-42c4-bb45-66db5bdb734c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:00.507401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75269c10-feda-42c4-bb45-66db5bdb734c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:00.507819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75269c10-feda-42c4-bb45-66db5bdb734c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:00.521124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:00.539038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-75269c10-feda-42c4-bb45-66db5bdb734c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046170 Aug 24 10:08:00.539649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 337/659] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:00 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:08:01.219716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:01.220349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:01.227274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:01.227711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:01.228178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:01.228861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:01.229315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:01.229737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:01.362687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:01.370993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-53561cad-d1c7-429e-afde-9a85f7524e3e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.155060 Aug 24 10:08:01.371663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 323/660] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:01 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:01.552448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f68be950-5eb7-40d0-9ffb-cb1546351175 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:01.554476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f68be950-5eb7-40d0-9ffb-cb1546351175 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:01.560992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f68be950-5eb7-40d0-9ffb-cb1546351175 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:01.561468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f68be950-5eb7-40d0-9ffb-cb1546351175 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:01.561898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f68be950-5eb7-40d0-9ffb-cb1546351175 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:01.562691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f68be950-5eb7-40d0-9ffb-cb1546351175 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:01.563141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f68be950-5eb7-40d0-9ffb-cb1546351175 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:01.563559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f68be950-5eb7-40d0-9ffb-cb1546351175 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:01.592292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f68be950-5eb7-40d0-9ffb-cb1546351175 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044480 Aug 24 10:08:01.592835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 338/661] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:02.393124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:02.393738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:02.400642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:02.401126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:02.401563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:02.402284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:02.402740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:02.403164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:02.530814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:02.538476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-95c81512-85d1-48bf-bd36-410b66d13e33 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.149135 Aug 24 10:08:02.539036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 324/662] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:02 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:02.605198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-25bc5ade-8f22-4e5d-9fdd-a2f91ef6edc0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:02.607347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-25bc5ade-8f22-4e5d-9fdd-a2f91ef6edc0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:02.613969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25bc5ade-8f22-4e5d-9fdd-a2f91ef6edc0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:02.614449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25bc5ade-8f22-4e5d-9fdd-a2f91ef6edc0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:02.614885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25bc5ade-8f22-4e5d-9fdd-a2f91ef6edc0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:02.615579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25bc5ade-8f22-4e5d-9fdd-a2f91ef6edc0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:02.616029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25bc5ade-8f22-4e5d-9fdd-a2f91ef6edc0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:02.616674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25bc5ade-8f22-4e5d-9fdd-a2f91ef6edc0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:02.646737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-25bc5ade-8f22-4e5d-9fdd-a2f91ef6edc0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046540 Aug 24 10:08:02.647291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 339/663] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:02 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:03.559494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:03.560120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:03.566470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:03.566951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:03.567447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:03.568167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:03.568620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:03.569039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:03.658829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fa4aec6e-b288-40e6-916a-9c8341aa6991 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:03.661609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fa4aec6e-b288-40e6-916a-9c8341aa6991 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:03.667973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa4aec6e-b288-40e6-916a-9c8341aa6991 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:03.668498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa4aec6e-b288-40e6-916a-9c8341aa6991 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:03.668939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa4aec6e-b288-40e6-916a-9c8341aa6991 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:03.669646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa4aec6e-b288-40e6-916a-9c8341aa6991 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:03.670106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa4aec6e-b288-40e6-916a-9c8341aa6991 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:03.670534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa4aec6e-b288-40e6-916a-9c8341aa6991 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:03.700217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fa4aec6e-b288-40e6-916a-9c8341aa6991 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045072 Aug 24 10:08:03.700541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 340/664] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:08:03.702862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:03.710998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-15fc0318-8ff5-4252-8979-1b6b99fdea66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.155352 Aug 24 10:08:03.711549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 325/665] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:03 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:04.716813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5271f46e-7a32-408d-a9fb-b1eda04bb014 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:04.718936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5271f46e-7a32-408d-a9fb-b1eda04bb014 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:04.727902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5271f46e-7a32-408d-a9fb-b1eda04bb014 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:04.728682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5271f46e-7a32-408d-a9fb-b1eda04bb014 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:04.729125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5271f46e-7a32-408d-a9fb-b1eda04bb014 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:04.729826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5271f46e-7a32-408d-a9fb-b1eda04bb014 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:04.730275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5271f46e-7a32-408d-a9fb-b1eda04bb014 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:04.730770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5271f46e-7a32-408d-a9fb-b1eda04bb014 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:04.731974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:04.732635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:04.740599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:04.741155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:04.741588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:04.742743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:04.742743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:04.743233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:04.764691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5271f46e-7a32-408d-a9fb-b1eda04bb014 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052429 Aug 24 10:08:04.765257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 341/666] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:04.874977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:04.878367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:04.883808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-17556ccd-9567-4d12-b848-850e74cb3b8a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.155752 Aug 24 10:08:04.884416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 326/667] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:05.777166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ebee9457-080a-4fbf-a928-6731d8c11b9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:05.779325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ebee9457-080a-4fbf-a928-6731d8c11b9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:05.786521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebee9457-080a-4fbf-a928-6731d8c11b9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:05.786926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebee9457-080a-4fbf-a928-6731d8c11b9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:05.787368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebee9457-080a-4fbf-a928-6731d8c11b9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:05.788103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebee9457-080a-4fbf-a928-6731d8c11b9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:05.788567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebee9457-080a-4fbf-a928-6731d8c11b9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:05.788996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebee9457-080a-4fbf-a928-6731d8c11b9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:05.818424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ebee9457-080a-4fbf-a928-6731d8c11b9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045398 Aug 24 10:08:05.818982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 342/668] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:05.904067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:05.904686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:05.912070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:05.912562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:05.912992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:05.913689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:05.914143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:05.914562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:06.049327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:06.059266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c20267f0-2213-48cb-bc29-e82d75c3e42d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.158790 Aug 24 10:08:06.060149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 327/669] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:06.923517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:08:06.924107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:08:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:08:06.940113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-41ab7f3a-a71f-4aad-a11f-3187b2626d6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:06.943001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-41ab7f3a-a71f-4aad-a11f-3187b2626d6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:06.950245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-41ab7f3a-a71f-4aad-a11f-3187b2626d6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:06.950597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-41ab7f3a-a71f-4aad-a11f-3187b2626d6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:06.951043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-41ab7f3a-a71f-4aad-a11f-3187b2626d6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:06.951746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-41ab7f3a-a71f-4aad-a11f-3187b2626d6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:06.952246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-41ab7f3a-a71f-4aad-a11f-3187b2626d6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:06.952677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-41ab7f3a-a71f-4aad-a11f-3187b2626d6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:06.982389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-41ab7f3a-a71f-4aad-a11f-3187b2626d6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.155513 Aug 24 10:08:06.982946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 343/670] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:07.081140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:07.081749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:07.088160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:07.088585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:07.089017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:07.089699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:07.090148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:07.090578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:07.226369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:07.234486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9c8e6dfd-0305-4cb7-9732-ecab2a932290 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.157175 Aug 24 10:08:07.235054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 328/671] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:07 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:07.995620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4eef50d3-25db-49cc-8930-109eb6ae4e10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:07.997800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4eef50d3-25db-49cc-8930-109eb6ae4e10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:08.004214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4eef50d3-25db-49cc-8930-109eb6ae4e10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:08.004702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4eef50d3-25db-49cc-8930-109eb6ae4e10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:08.005153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4eef50d3-25db-49cc-8930-109eb6ae4e10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:08.005844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4eef50d3-25db-49cc-8930-109eb6ae4e10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:08.006305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4eef50d3-25db-49cc-8930-109eb6ae4e10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:08.006735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4eef50d3-25db-49cc-8930-109eb6ae4e10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:08.035316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4eef50d3-25db-49cc-8930-109eb6ae4e10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043666 Aug 24 10:08:08.035794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 344/672] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:08.255947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:08.256584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:08.262667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:08.263149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:08.263572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:08.264281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:08.264726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:08.265143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:08.394421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:08.402612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-184cfa4b-715a-426a-8efd-79cd1f401d3a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.150547 Aug 24 10:08:08.403214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 329/673] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:08 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:08.594957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-ee903998-9a5c-40f5-bc3d-35edcc872a30 req-268e6c7f-180c-4e22-b47f-4dcb02b15419 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0c601f9d-f26b-4095-a00c-53c422163d31", "name": "network-vif-unplugged", "status": "completed", "tag": "daf8264d-e786-4857-bf7e-3e865d7b48a2"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:08:08.605961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-ee903998-9a5c-40f5-bc3d-35edcc872a30 req-268e6c7f-180c-4e22-b47f-4dcb02b15419 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:08.606451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-ee903998-9a5c-40f5-bc3d-35edcc872a30 req-268e6c7f-180c-4e22-b47f-4dcb02b15419 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:08.606886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-ee903998-9a5c-40f5-bc3d-35edcc872a30 req-268e6c7f-180c-4e22-b47f-4dcb02b15419 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:08.618407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-ee903998-9a5c-40f5-bc3d-35edcc872a30 req-268e6c7f-180c-4e22-b47f-4dcb02b15419 service nova] Creating event network-vif-unplugged:daf8264d-e786-4857-bf7e-3e865d7b48a2 for instance 0c601f9d-f26b-4095-a00c-53c422163d31 on hv-cinder-85427 Aug 24 10:08:08.623365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-ee903998-9a5c-40f5-bc3d-35edcc872a30 req-268e6c7f-180c-4e22-b47f-4dcb02b15419 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032393 Aug 24 10:08:08.624022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 345/674] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:08:08 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:08:09.047436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5dc42928-e16a-4aeb-8fe8-a616e330af20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:09.049597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5dc42928-e16a-4aeb-8fe8-a616e330af20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:09.056021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dc42928-e16a-4aeb-8fe8-a616e330af20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:09.056534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dc42928-e16a-4aeb-8fe8-a616e330af20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:09.056969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dc42928-e16a-4aeb-8fe8-a616e330af20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:09.057658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dc42928-e16a-4aeb-8fe8-a616e330af20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:09.058119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dc42928-e16a-4aeb-8fe8-a616e330af20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:09.058543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dc42928-e16a-4aeb-8fe8-a616e330af20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:09.087282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5dc42928-e16a-4aeb-8fe8-a616e330af20 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043887 Aug 24 10:08:09.087835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 330/675] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:09 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:09.425522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:09.426240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:09.432371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:09.432863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:09.433304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:09.433995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:09.434454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:09.434886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:09.613543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:09.621971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3aac1d73-d32b-4b4a-83b9-0c49db7f349b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.201145 Aug 24 10:08:09.622550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 346/676] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:09 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:10.100117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9ee601ca-dfa5-4a06-98cd-c83826f8b2fd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:10.102487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9ee601ca-dfa5-4a06-98cd-c83826f8b2fd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:10.108817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ee601ca-dfa5-4a06-98cd-c83826f8b2fd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:10.109298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ee601ca-dfa5-4a06-98cd-c83826f8b2fd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:10.109727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ee601ca-dfa5-4a06-98cd-c83826f8b2fd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:10.110410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ee601ca-dfa5-4a06-98cd-c83826f8b2fd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:10.110854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ee601ca-dfa5-4a06-98cd-c83826f8b2fd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:10.111271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ee601ca-dfa5-4a06-98cd-c83826f8b2fd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:10.140997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9ee601ca-dfa5-4a06-98cd-c83826f8b2fd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045335 Aug 24 10:08:10.141550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 331/677] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:10 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:10.642804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:10.643338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:10.650605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:10.651093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:10.651532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:10.652254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:10.652715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:10.653220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:10.794132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:10.801744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4f3eb95a-f883-4c89-8946-749b53731120 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.162915 Aug 24 10:08:10.802300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 347/678] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:10 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:11.152919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a121c4c6-c367-48ab-aed9-17da1b8d0c46 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:11.155087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a121c4c6-c367-48ab-aed9-17da1b8d0c46 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:11.161713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a121c4c6-c367-48ab-aed9-17da1b8d0c46 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:11.162289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a121c4c6-c367-48ab-aed9-17da1b8d0c46 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:11.162724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a121c4c6-c367-48ab-aed9-17da1b8d0c46 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:11.163415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a121c4c6-c367-48ab-aed9-17da1b8d0c46 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:11.163874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a121c4c6-c367-48ab-aed9-17da1b8d0c46 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:11.164321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a121c4c6-c367-48ab-aed9-17da1b8d0c46 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:11.193876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a121c4c6-c367-48ab-aed9-17da1b8d0c46 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044894 Aug 24 10:08:11.194423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 332/679] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:11 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:11.822225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:11.822834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:11.829318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:11.829800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:11.830234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:11.830920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:11.831375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:11.831800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:11.958899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:11.966862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ad174df8-f738-47d3-a0eb-e5f09c121a80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.148443 Aug 24 10:08:11.967426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 348/680] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:11 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:12.038865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:08:12.039057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:08:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:08:12.206164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a3a38202-256d-426e-bdda-f80be93150f5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:12.208952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a3a38202-256d-426e-bdda-f80be93150f5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:12.215278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3a38202-256d-426e-bdda-f80be93150f5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:12.215754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3a38202-256d-426e-bdda-f80be93150f5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:12.216231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3a38202-256d-426e-bdda-f80be93150f5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:12.216902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3a38202-256d-426e-bdda-f80be93150f5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:12.217349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3a38202-256d-426e-bdda-f80be93150f5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:12.217765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3a38202-256d-426e-bdda-f80be93150f5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:12.251136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a3a38202-256d-426e-bdda-f80be93150f5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048912 Aug 24 10:08:12.251681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 333/681] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:12 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:12.988783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:12.990042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:12.997152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:12.997635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:12.998070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:12.998758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:12.999213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:12.999645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:13.138688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:13.146535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-89ca65f6-bbdb-4687-b7c3-7f5c2e476f68 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.161605 Aug 24 10:08:13.147091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 349/682] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:12 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:13.263297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-80073f23-8b47-4fce-8dbd-cc771516913a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:13.265365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-80073f23-8b47-4fce-8dbd-cc771516913a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:13.271819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80073f23-8b47-4fce-8dbd-cc771516913a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:13.272323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80073f23-8b47-4fce-8dbd-cc771516913a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:13.272756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80073f23-8b47-4fce-8dbd-cc771516913a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:13.273437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80073f23-8b47-4fce-8dbd-cc771516913a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:13.273886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80073f23-8b47-4fce-8dbd-cc771516913a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:13.274306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80073f23-8b47-4fce-8dbd-cc771516913a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:13.303314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-80073f23-8b47-4fce-8dbd-cc771516913a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044030 Aug 24 10:08:13.303849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 334/683] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:14.168467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:14.169001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:14.175286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:14.175759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:14.176226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:14.176937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:14.177388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:14.177806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:14.304898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:14.312461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0ddb9001-8f1d-4b40-b53e-4c37f28e1c09 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.147950 Aug 24 10:08:14.313025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 350/684] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:14 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:14.315434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9682375d-6462-4d65-9985-5de885873730 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:14.317645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9682375d-6462-4d65-9985-5de885873730 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:14.320982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=52 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:14.325047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9682375d-6462-4d65-9985-5de885873730 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:14.325532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9682375d-6462-4d65-9985-5de885873730 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:14.325970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9682375d-6462-4d65-9985-5de885873730 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:14.326664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9682375d-6462-4d65-9985-5de885873730 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:14.327121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9682375d-6462-4d65-9985-5de885873730 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:14.327544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9682375d-6462-4d65-9985-5de885873730 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:14.361199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9682375d-6462-4d65-9985-5de885873730 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049840 Aug 24 10:08:14.361768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 335/685] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:14 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:08:15.333370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:15.333981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:15.340846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:15.341479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:15.341911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:15.342592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:15.343069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:15.343489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:15.373511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4fe99cb6-0343-42dc-a07f-731ad590c504 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:15.375728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4fe99cb6-0343-42dc-a07f-731ad590c504 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:15.382774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fe99cb6-0343-42dc-a07f-731ad590c504 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:15.383265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fe99cb6-0343-42dc-a07f-731ad590c504 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:15.383986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fe99cb6-0343-42dc-a07f-731ad590c504 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:15.385210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fe99cb6-0343-42dc-a07f-731ad590c504 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:15.386251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fe99cb6-0343-42dc-a07f-731ad590c504 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:15.386933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fe99cb6-0343-42dc-a07f-731ad590c504 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:15.431061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4fe99cb6-0343-42dc-a07f-731ad590c504 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061420 Aug 24 10:08:15.431645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 336/686] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:08:15.479197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:15.486926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a24ed933-58a8-41bb-bc84-595bfca393ee tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.157418 Aug 24 10:08:15.487484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 351/687] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:15 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:16.445047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3e95b310-dbe3-4413-b4f4-d3d6cce07ca4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:16.447161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3e95b310-dbe3-4413-b4f4-d3d6cce07ca4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:16.453507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e95b310-dbe3-4413-b4f4-d3d6cce07ca4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:16.453986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e95b310-dbe3-4413-b4f4-d3d6cce07ca4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:16.454421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e95b310-dbe3-4413-b4f4-d3d6cce07ca4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:16.455108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e95b310-dbe3-4413-b4f4-d3d6cce07ca4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:16.455567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e95b310-dbe3-4413-b4f4-d3d6cce07ca4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:16.455994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e95b310-dbe3-4413-b4f4-d3d6cce07ca4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:16.487242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3e95b310-dbe3-4413-b4f4-d3d6cce07ca4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046209 Aug 24 10:08:16.487784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 337/688] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:16 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:16.508320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:16.508935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:16.515202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:16.515684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:16.516154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:16.516840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:16.517294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:16.517718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:16.647768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:16.655503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-77154042-29ee-4dff-9596-6fc4d3bf9853 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.151098 Aug 24 10:08:16.656284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 352/689] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:16 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:17.499558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9fa9903d-4641-4b24-9a26-6c90dc865b59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:17.501689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9fa9903d-4641-4b24-9a26-6c90dc865b59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:17.508574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fa9903d-4641-4b24-9a26-6c90dc865b59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:17.509054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fa9903d-4641-4b24-9a26-6c90dc865b59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:17.509489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fa9903d-4641-4b24-9a26-6c90dc865b59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:17.510178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fa9903d-4641-4b24-9a26-6c90dc865b59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:17.510632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fa9903d-4641-4b24-9a26-6c90dc865b59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:17.511058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fa9903d-4641-4b24-9a26-6c90dc865b59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:17.539841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9fa9903d-4641-4b24-9a26-6c90dc865b59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044283 Aug 24 10:08:17.540410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 338/690] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:17.676287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:17.676959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:17.683256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:17.683725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:17.684218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:17.684904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:17.685357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:17.685781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:17.821403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:17.829054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9fcbb463-ebda-4a91-b327-b6cb6cc01eab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.156672 Aug 24 10:08:17.829604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 353/691] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:17 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:18.552955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a1566c08-0ff8-4f0d-a48c-5e337409dc39 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:18.555063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a1566c08-0ff8-4f0d-a48c-5e337409dc39 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:18.561776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a1566c08-0ff8-4f0d-a48c-5e337409dc39 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:18.562256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a1566c08-0ff8-4f0d-a48c-5e337409dc39 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:18.562717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a1566c08-0ff8-4f0d-a48c-5e337409dc39 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:18.563409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a1566c08-0ff8-4f0d-a48c-5e337409dc39 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:18.563860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a1566c08-0ff8-4f0d-a48c-5e337409dc39 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:18.564313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a1566c08-0ff8-4f0d-a48c-5e337409dc39 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:18.593098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a1566c08-0ff8-4f0d-a48c-5e337409dc39 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044745 Aug 24 10:08:18.593651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 339/692] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:18 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:18.850258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:18.850900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:18.857433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:18.857915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:18.858351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:18.859030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:18.859482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:18.859898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:18.991325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:18.993699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:19.000466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-33eabe09-7985-42fc-b3cf-b9d2e0c9602e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.154572 Aug 24 10:08:19.001034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 354/693] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:18 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:19.606016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-86a7cbbf-4ad3-4ae1-81d2-5f90bdd16e0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:19.608152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-86a7cbbf-4ad3-4ae1-81d2-5f90bdd16e0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:19.614683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a7cbbf-4ad3-4ae1-81d2-5f90bdd16e0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:19.615156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a7cbbf-4ad3-4ae1-81d2-5f90bdd16e0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:19.615581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a7cbbf-4ad3-4ae1-81d2-5f90bdd16e0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:19.616282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a7cbbf-4ad3-4ae1-81d2-5f90bdd16e0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:19.616730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a7cbbf-4ad3-4ae1-81d2-5f90bdd16e0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:19.617140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a7cbbf-4ad3-4ae1-81d2-5f90bdd16e0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:19.646751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-86a7cbbf-4ad3-4ae1-81d2-5f90bdd16e0c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044989 Aug 24 10:08:19.647296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 340/694] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:19.725213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1905674680"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:08:19.725751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:19.732074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:19.732564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:19.733001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:19.733689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:19.734409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:19.734832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:19.858362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.network.security_group_api [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Removing security group cb10e7ee-2c1d-45b7-9519-9ff73d9d547c from port 98183aae-275c-45c1-b9fc-a7d038275f2d Aug 24 10:08:20.022001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:20.022505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:20.028439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:20.028928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:20.029374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:20.030069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:20.030539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:20.030951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:20.171873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:20.181322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5e13a1c2-be07-42a2-ae75-00dcc1fdb90e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.163315 Aug 24 10:08:20.181908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 341/695] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:20 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:20.186822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6133b481-ecee-441d-8a65-75cad18fa723 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "POST /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/action" status: 202 len: 0 microversion: 2.1 time: 0.466172 Aug 24 10:08:20.190755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 355/696] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:08:19 2022] POST /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/action => generated 0 bytes in 468 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:08:20.196684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-4d852379-3709-4adb-9ccf-a76e20a22095 req-a097cdcf-5626-47eb-a87a-62804927d91a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "987654b5-b3b6-434c-a284-d630ab8ab4ad", "tag": "98183aae-275c-45c1-b9fc-a7d038275f2d"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:08:20.207094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-4d852379-3709-4adb-9ccf-a76e20a22095 req-a097cdcf-5626-47eb-a87a-62804927d91a service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:20.207831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-4d852379-3709-4adb-9ccf-a76e20a22095 req-a097cdcf-5626-47eb-a87a-62804927d91a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:20.208980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-4d852379-3709-4adb-9ccf-a76e20a22095 req-a097cdcf-5626-47eb-a87a-62804927d91a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:20.221221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-4d852379-3709-4adb-9ccf-a76e20a22095 req-a097cdcf-5626-47eb-a87a-62804927d91a service nova] Creating event network-changed:98183aae-275c-45c1-b9fc-a7d038275f2d for instance 987654b5-b3b6-434c-a284-d630ab8ab4ad on hv-cinder-85427 Aug 24 10:08:20.227715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-4d852379-3709-4adb-9ccf-a76e20a22095 req-a097cdcf-5626-47eb-a87a-62804927d91a service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034392 Aug 24 10:08:20.228841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 342/697] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:08:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:08:20.658906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5f73fb5d-f985-40da-a6f7-caddfb5e3779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:20.661666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5f73fb5d-f985-40da-a6f7-caddfb5e3779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:20.669099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f73fb5d-f985-40da-a6f7-caddfb5e3779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:20.669785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f73fb5d-f985-40da-a6f7-caddfb5e3779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:20.670544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f73fb5d-f985-40da-a6f7-caddfb5e3779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:20.671251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f73fb5d-f985-40da-a6f7-caddfb5e3779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:20.671704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f73fb5d-f985-40da-a6f7-caddfb5e3779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:20.672327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f73fb5d-f985-40da-a6f7-caddfb5e3779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:20.702958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5f73fb5d-f985-40da-a6f7-caddfb5e3779 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048003 Aug 24 10:08:20.703523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 356/698] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:20 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:21.202842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:21.203534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:21.209946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:21.210446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:21.210877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:21.211574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:21.212031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:21.212478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:21.470311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:21.478706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8c2f6229-1871-41c0-9558-d897f45fb784 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.280051 Aug 24 10:08:21.479265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 343/699] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:21 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:21.715294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5131a2f0-5d11-4b4c-bd03-7efb9f35f088 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:21.717535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5131a2f0-5d11-4b4c-bd03-7efb9f35f088 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:21.726122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5131a2f0-5d11-4b4c-bd03-7efb9f35f088 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:21.726724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5131a2f0-5d11-4b4c-bd03-7efb9f35f088 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:21.727761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5131a2f0-5d11-4b4c-bd03-7efb9f35f088 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:21.728857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5131a2f0-5d11-4b4c-bd03-7efb9f35f088 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:21.729329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5131a2f0-5d11-4b4c-bd03-7efb9f35f088 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:21.729759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5131a2f0-5d11-4b4c-bd03-7efb9f35f088 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:21.739997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:21.740522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:21.746729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:21.747247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:21.747678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:21.748458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:21.748906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:21.749323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:21.776558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5131a2f0-5d11-4b4c-bd03-7efb9f35f088 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065099 Aug 24 10:08:21.777452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 357/700] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:21 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:21.780314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:21.785104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-0f36b90e-ad16-4526-b431-c57dbe654d5a req-098f9a50-e373-48bf-befc-9026ab537751 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "987654b5-b3b6-434c-a284-d630ab8ab4ad", "tag": "98183aae-275c-45c1-b9fc-a7d038275f2d"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:08:21.796425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-0f36b90e-ad16-4526-b431-c57dbe654d5a req-098f9a50-e373-48bf-befc-9026ab537751 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:21.796907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-0f36b90e-ad16-4526-b431-c57dbe654d5a req-098f9a50-e373-48bf-befc-9026ab537751 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:21.797338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-0f36b90e-ad16-4526-b431-c57dbe654d5a req-098f9a50-e373-48bf-befc-9026ab537751 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:21.810669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-0f36b90e-ad16-4526-b431-c57dbe654d5a req-098f9a50-e373-48bf-befc-9026ab537751 service nova] Creating event network-changed:98183aae-275c-45c1-b9fc-a7d038275f2d for instance 987654b5-b3b6-434c-a284-d630ab8ab4ad on hv-cinder-85427 Aug 24 10:08:21.816225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-0f36b90e-ad16-4526-b431-c57dbe654d5a req-098f9a50-e373-48bf-befc-9026ab537751 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037380 Aug 24 10:08:21.816831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 358/701] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:08:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:08:22.500259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:22.500879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:22.508169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:22.508658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:22.509091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:22.510040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:22.510492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:22.510912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:22.695251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:22.703195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6c1790e2-4168-41c7-bfcc-cd649d33898e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.207478 Aug 24 10:08:22.703782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 359/702] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:22 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:22.790032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-af90bb5f-ec55-47c5-bae5-774b4fba64a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:22.793164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-af90bb5f-ec55-47c5-bae5-774b4fba64a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:22.799737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af90bb5f-ec55-47c5-bae5-774b4fba64a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:22.800263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af90bb5f-ec55-47c5-bae5-774b4fba64a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:22.800734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af90bb5f-ec55-47c5-bae5-774b4fba64a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:22.801646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af90bb5f-ec55-47c5-bae5-774b4fba64a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:22.802743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af90bb5f-ec55-47c5-bae5-774b4fba64a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:22.803171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af90bb5f-ec55-47c5-bae5-774b4fba64a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:22.833015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-af90bb5f-ec55-47c5-bae5-774b4fba64a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047475 Aug 24 10:08:22.833577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 360/703] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:22 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:23.725013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:23.725627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:23.731862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:23.732445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:23.732887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:23.733574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:23.734027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:23.734450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:23.867330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:23.875342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a7778126-766c-4d07-83bb-a260c8e7ec84 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.154690 Aug 24 10:08:23.875913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 361/704] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:23 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:23.880502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b03b70c5-ddf8-4e70-88f8-15dbd4654fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:23.882607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b03b70c5-ddf8-4e70-88f8-15dbd4654fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:23.888252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b03b70c5-ddf8-4e70-88f8-15dbd4654fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:23.888740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b03b70c5-ddf8-4e70-88f8-15dbd4654fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:23.889179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b03b70c5-ddf8-4e70-88f8-15dbd4654fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:23.889871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b03b70c5-ddf8-4e70-88f8-15dbd4654fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:23.890334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b03b70c5-ddf8-4e70-88f8-15dbd4654fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:23.890760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b03b70c5-ddf8-4e70-88f8-15dbd4654fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:23.921784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b03b70c5-ddf8-4e70-88f8-15dbd4654fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044827 Aug 24 10:08:23.922341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 362/705] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:23 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:24.896415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:24.897068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:24.903459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:24.903945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:24.904468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:24.905109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:24.905567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:24.905997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:25.035297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:25.043801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7e3e6b0b-d0ba-4d1a-b522-636279cf0b21 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.151209 Aug 24 10:08:25.044479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 363/706] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:24 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:25.049023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-39dd64d5-ebbf-4996-b729-ee8d326f23bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:25.051130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-39dd64d5-ebbf-4996-b729-ee8d326f23bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:25.057286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39dd64d5-ebbf-4996-b729-ee8d326f23bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:25.057797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39dd64d5-ebbf-4996-b729-ee8d326f23bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:25.058235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39dd64d5-ebbf-4996-b729-ee8d326f23bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:25.058929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39dd64d5-ebbf-4996-b729-ee8d326f23bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:25.059496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39dd64d5-ebbf-4996-b729-ee8d326f23bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:25.059930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39dd64d5-ebbf-4996-b729-ee8d326f23bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:25.088151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-39dd64d5-ebbf-4996-b729-ee8d326f23bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042649 Aug 24 10:08:25.088696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 364/707] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:25 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:26.066176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:26.066812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:26.073217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:26.073705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:26.074146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:26.074898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:26.075357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:26.075787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:26.212840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:26.220437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-589a8fd4-ede1-4223-b9fd-33e986590463 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.158358 Aug 24 10:08:26.221003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 365/708] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:26 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:26.226064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4066edaf-a62e-4a6d-92ae-6d0befe1226c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:26.228427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4066edaf-a62e-4a6d-92ae-6d0befe1226c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:26.233951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4066edaf-a62e-4a6d-92ae-6d0befe1226c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:26.234430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4066edaf-a62e-4a6d-92ae-6d0befe1226c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:26.234859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4066edaf-a62e-4a6d-92ae-6d0befe1226c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:26.235544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4066edaf-a62e-4a6d-92ae-6d0befe1226c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:26.235995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4066edaf-a62e-4a6d-92ae-6d0befe1226c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:26.236442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4066edaf-a62e-4a6d-92ae-6d0befe1226c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:26.264820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4066edaf-a62e-4a6d-92ae-6d0befe1226c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042821 Aug 24 10:08:26.265364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 366/709] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:26 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:27.242437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:27.244333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:27.248071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=68 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:27.251984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:27.252486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:27.252922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:27.254110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:27.254110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:27.254446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:27.382518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:27.390467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0b60118b-e31f-424c-bc10-592113cc12b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.151976 Aug 24 10:08:27.391037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 367/710] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:27 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:27.395657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2c79cf4c-1699-4baa-bc9c-ddd1b8d2e77e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:27.397807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2c79cf4c-1699-4baa-bc9c-ddd1b8d2e77e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:27.403466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c79cf4c-1699-4baa-bc9c-ddd1b8d2e77e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:27.403947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c79cf4c-1699-4baa-bc9c-ddd1b8d2e77e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:27.404430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c79cf4c-1699-4baa-bc9c-ddd1b8d2e77e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:27.405121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c79cf4c-1699-4baa-bc9c-ddd1b8d2e77e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:27.405581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c79cf4c-1699-4baa-bc9c-ddd1b8d2e77e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:27.406006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c79cf4c-1699-4baa-bc9c-ddd1b8d2e77e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:27.435444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2c79cf4c-1699-4baa-bc9c-ddd1b8d2e77e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043401 Aug 24 10:08:27.435990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 368/711] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:27 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:28.412286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:28.412961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:28.419301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:28.419792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:28.420264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:28.420957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:28.421415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:28.421841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:28.555483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:28.563233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-44be704c-3a7b-4e71-9c0b-ba3117699ca4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.155520 Aug 24 10:08:28.563794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 369/712] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:28 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:28.569033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-71316ed3-b18a-48c5-9169-80d759a5d19b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:28.571140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-71316ed3-b18a-48c5-9169-80d759a5d19b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:28.576803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71316ed3-b18a-48c5-9169-80d759a5d19b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:28.577290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71316ed3-b18a-48c5-9169-80d759a5d19b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:28.577727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71316ed3-b18a-48c5-9169-80d759a5d19b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:28.578416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71316ed3-b18a-48c5-9169-80d759a5d19b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:28.578866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71316ed3-b18a-48c5-9169-80d759a5d19b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:28.579321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71316ed3-b18a-48c5-9169-80d759a5d19b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:28.609184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-71316ed3-b18a-48c5-9169-80d759a5d19b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044280 Aug 24 10:08:28.609736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 370/713] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:28 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:29.585432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:29.586082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:29.592388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:29.592870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:29.593307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:29.593990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:29.594441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:29.594865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:29.726918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:29.734882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-42221d0b-c313-441d-a948-492c4196b8b9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.153990 Aug 24 10:08:29.735446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 371/714] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:29 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:29.740546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8d27fd6a-02d5-4183-a9d5-e11213857db9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:29.742733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8d27fd6a-02d5-4183-a9d5-e11213857db9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:29.748353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d27fd6a-02d5-4183-a9d5-e11213857db9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:29.748835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d27fd6a-02d5-4183-a9d5-e11213857db9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:29.749272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d27fd6a-02d5-4183-a9d5-e11213857db9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:29.749958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d27fd6a-02d5-4183-a9d5-e11213857db9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:29.750411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d27fd6a-02d5-4183-a9d5-e11213857db9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:29.750832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d27fd6a-02d5-4183-a9d5-e11213857db9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:29.764034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:29.780732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8d27fd6a-02d5-4183-a9d5-e11213857db9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044271 Aug 24 10:08:29.781283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 372/715] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:30.756885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:30.757544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:30.763921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:30.764439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:30.764882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:30.765580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:30.766037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:30.766463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:30.901514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:30.909273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5e981ae7-1628-4a7f-9a9b-698f41e4585a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.156210 Aug 24 10:08:30.909768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 373/716] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:30 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:30.914900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4a3e616d-e163-4fb8-8bcb-2173f81b9317 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:30.917048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4a3e616d-e163-4fb8-8bcb-2173f81b9317 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:30.922638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a3e616d-e163-4fb8-8bcb-2173f81b9317 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:30.923123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a3e616d-e163-4fb8-8bcb-2173f81b9317 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:30.923556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a3e616d-e163-4fb8-8bcb-2173f81b9317 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:30.924276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a3e616d-e163-4fb8-8bcb-2173f81b9317 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:30.924738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a3e616d-e163-4fb8-8bcb-2173f81b9317 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:30.925159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a3e616d-e163-4fb8-8bcb-2173f81b9317 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:30.953673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4a3e616d-e163-4fb8-8bcb-2173f81b9317 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042901 Aug 24 10:08:30.954221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 374/717] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:31.744403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=19 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:31.813683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:31.931036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:31.931644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:31.938063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:31.938547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:31.938986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:31.939673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:31.940163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:31.940600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:32.074067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:32.081926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e7c571f5-cefa-4334-858e-6f31624bcce0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.155224 Aug 24 10:08:32.082493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 375/718] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:31 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:32.087483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2c379af7-d9fe-4be1-bdec-4398c3414e4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:32.089598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2c379af7-d9fe-4be1-bdec-4398c3414e4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:32.095109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c379af7-d9fe-4be1-bdec-4398c3414e4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:32.095598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c379af7-d9fe-4be1-bdec-4398c3414e4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:32.096060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c379af7-d9fe-4be1-bdec-4398c3414e4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:32.096751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c379af7-d9fe-4be1-bdec-4398c3414e4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:32.097211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c379af7-d9fe-4be1-bdec-4398c3414e4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:32.097646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c379af7-d9fe-4be1-bdec-4398c3414e4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:32.126685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2c379af7-d9fe-4be1-bdec-4398c3414e4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043205 Aug 24 10:08:32.127234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 376/719] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:32 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:33.103395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:33.104164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:33.110745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:33.111242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:33.111770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:33.112515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:33.112975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:33.113469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:33.257682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:33.265385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7e6faf04-584d-48ec-8cd9-4eb322210f55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.166555 Aug 24 10:08:33.265943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 377/720] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:33 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:33.270500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1e0ed0f6-d026-4fd9-ac29-6bebe4094a2d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:33.272735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1e0ed0f6-d026-4fd9-ac29-6bebe4094a2d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:33.278258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e0ed0f6-d026-4fd9-ac29-6bebe4094a2d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:33.278734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e0ed0f6-d026-4fd9-ac29-6bebe4094a2d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:33.279165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e0ed0f6-d026-4fd9-ac29-6bebe4094a2d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:33.279846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e0ed0f6-d026-4fd9-ac29-6bebe4094a2d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:33.280318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e0ed0f6-d026-4fd9-ac29-6bebe4094a2d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:33.280736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e0ed0f6-d026-4fd9-ac29-6bebe4094a2d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:33.311990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1e0ed0f6-d026-4fd9-ac29-6bebe4094a2d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045053 Aug 24 10:08:33.312553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 378/721] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:34.286667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:34.287382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:34.293689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:34.294175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:34.294624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:34.295311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:34.295772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:34.296230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:34.425620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:34.433860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4a5a2108-7801-4df6-99f4-92b423f73dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.151091 Aug 24 10:08:34.434415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 379/722] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:34 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:34.438962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-70bc294b-3dd6-4386-89c8-7157ff23f9e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:34.441035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-70bc294b-3dd6-4386-89c8-7157ff23f9e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:34.447449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70bc294b-3dd6-4386-89c8-7157ff23f9e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:34.447931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70bc294b-3dd6-4386-89c8-7157ff23f9e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:34.448419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70bc294b-3dd6-4386-89c8-7157ff23f9e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:34.449105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70bc294b-3dd6-4386-89c8-7157ff23f9e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:34.449555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70bc294b-3dd6-4386-89c8-7157ff23f9e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:34.449974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70bc294b-3dd6-4386-89c8-7157ff23f9e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:34.478848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-70bc294b-3dd6-4386-89c8-7157ff23f9e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043446 Aug 24 10:08:34.479394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 380/723] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:34 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:35.455675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:35.456365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:35.462990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:35.463482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:35.463927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:35.464645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:35.465134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:35.465563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:35.596685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:35.604742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-26d39de4-6bff-4416-bbe0-2da95e5bbf6d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.153152 Aug 24 10:08:35.605299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 381/724] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:35 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:35.609852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-59c9c29c-1fce-4a9e-b5ca-b2b7ac0caa81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:35.612037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-59c9c29c-1fce-4a9e-b5ca-b2b7ac0caa81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:35.617960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c9c29c-1fce-4a9e-b5ca-b2b7ac0caa81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:35.618520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c9c29c-1fce-4a9e-b5ca-b2b7ac0caa81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:35.618961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c9c29c-1fce-4a9e-b5ca-b2b7ac0caa81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:35.619653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c9c29c-1fce-4a9e-b5ca-b2b7ac0caa81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:35.620140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c9c29c-1fce-4a9e-b5ca-b2b7ac0caa81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:35.620591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c9c29c-1fce-4a9e-b5ca-b2b7ac0caa81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:35.650132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-59c9c29c-1fce-4a9e-b5ca-b2b7ac0caa81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043828 Aug 24 10:08:35.650675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 382/725] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:35 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:36.625856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:36.626519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:36.632981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:36.633240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:36.633669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:36.634347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:36.634793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:36.635212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:36.763608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:36.772406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-937b8901-0489-48ee-8a79-9fec1e84df02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.150334 Aug 24 10:08:36.772993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 383/726] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:36 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:36.777955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-60359efc-2cdb-4d35-af5a-f94d8ff4c7ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:36.780074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-60359efc-2cdb-4d35-af5a-f94d8ff4c7ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:36.785827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60359efc-2cdb-4d35-af5a-f94d8ff4c7ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:36.786909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60359efc-2cdb-4d35-af5a-f94d8ff4c7ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:36.787406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60359efc-2cdb-4d35-af5a-f94d8ff4c7ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:36.788157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60359efc-2cdb-4d35-af5a-f94d8ff4c7ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:36.788620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60359efc-2cdb-4d35-af5a-f94d8ff4c7ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:36.789051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60359efc-2cdb-4d35-af5a-f94d8ff4c7ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:36.817873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-60359efc-2cdb-4d35-af5a-f94d8ff4c7ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043590 Aug 24 10:08:36.818434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 384/727] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:36 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:37.793288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:37.793935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:37.800369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:37.800851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:37.801287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:37.802628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:37.803100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:37.803521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:37.934941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:37.942573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7d25575c-1899-4c7d-a081-b541f1d31361 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.153122 Aug 24 10:08:37.943133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 385/728] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:37 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:37.947774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5ce1d81f-1c90-4e9c-831d-b005d6be4bf7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:37.949886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5ce1d81f-1c90-4e9c-831d-b005d6be4bf7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:37.955310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ce1d81f-1c90-4e9c-831d-b005d6be4bf7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:37.955790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ce1d81f-1c90-4e9c-831d-b005d6be4bf7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:37.956260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ce1d81f-1c90-4e9c-831d-b005d6be4bf7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:37.956971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ce1d81f-1c90-4e9c-831d-b005d6be4bf7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:37.957423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ce1d81f-1c90-4e9c-831d-b005d6be4bf7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:37.957846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ce1d81f-1c90-4e9c-831d-b005d6be4bf7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:37.986590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5ce1d81f-1c90-4e9c-831d-b005d6be4bf7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042475 Aug 24 10:08:37.987128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 386/729] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:38.963176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:38.963864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:38.970146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:38.970626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:38.971060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:38.971740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:38.972214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:38.972638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:39.103204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:39.110770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-98f9f633-3bf1-4468-9fa4-09f0a9322ed3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.151429 Aug 24 10:08:39.111334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 387/730] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:38 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:08:39.115858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e259dc34-3742-4faf-8399-76915a07e786 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:39.118053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e259dc34-3742-4faf-8399-76915a07e786 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:39.123639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e259dc34-3742-4faf-8399-76915a07e786 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:39.124158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e259dc34-3742-4faf-8399-76915a07e786 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:39.124598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e259dc34-3742-4faf-8399-76915a07e786 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:39.125286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e259dc34-3742-4faf-8399-76915a07e786 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:39.125741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e259dc34-3742-4faf-8399-76915a07e786 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:39.126278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e259dc34-3742-4faf-8399-76915a07e786 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:39.155133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e259dc34-3742-4faf-8399-76915a07e786 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042816 Aug 24 10:08:39.155685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 388/731] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:39 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:40.132328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:40.132990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:40.139356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:40.139837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:40.140312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:40.140999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:40.141450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:40.141874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:40.145770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:40.274115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:40.281795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2ca125a9-8dff-4c34-9e10-19d24e02b68d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.153383 Aug 24 10:08:40.282354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 389/732] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:40 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:40.287381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-49f44cf2-101e-42d3-8a35-bb1c18fb585a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:40.289482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-49f44cf2-101e-42d3-8a35-bb1c18fb585a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:40.295302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f44cf2-101e-42d3-8a35-bb1c18fb585a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:40.295780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f44cf2-101e-42d3-8a35-bb1c18fb585a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:40.296457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f44cf2-101e-42d3-8a35-bb1c18fb585a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:40.296935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f44cf2-101e-42d3-8a35-bb1c18fb585a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:40.297382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f44cf2-101e-42d3-8a35-bb1c18fb585a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:40.297800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f44cf2-101e-42d3-8a35-bb1c18fb585a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:40.326853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-49f44cf2-101e-42d3-8a35-bb1c18fb585a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043506 Aug 24 10:08:40.327393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 390/733] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:41.302744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:41.303415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:41.309788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:41.310267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:41.310700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:41.311385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:41.311839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:41.312292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:41.440742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:41.448868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b71254c5-2aa2-49b4-bd24-8e4207c3f982 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.150087 Aug 24 10:08:41.449427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 391/734] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:41 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:41.453922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-258d4c70-991c-4590-add9-9ec1a4611ea3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:41.455999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-258d4c70-991c-4590-add9-9ec1a4611ea3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:41.462256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-258d4c70-991c-4590-add9-9ec1a4611ea3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:41.462739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-258d4c70-991c-4590-add9-9ec1a4611ea3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:41.463175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-258d4c70-991c-4590-add9-9ec1a4611ea3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:41.463860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-258d4c70-991c-4590-add9-9ec1a4611ea3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:41.464344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-258d4c70-991c-4590-add9-9ec1a4611ea3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:41.464768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-258d4c70-991c-4590-add9-9ec1a4611ea3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:41.493223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-258d4c70-991c-4590-add9-9ec1a4611ea3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042792 Aug 24 10:08:41.493769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 392/735] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:41 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:42.469358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:42.470028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:42.476877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:42.477360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:42.477802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:42.478771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:42.479215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:42.479641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:42.608491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:42.616061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5c9a8433-e11a-47ce-951c-e68a387da2ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.150557 Aug 24 10:08:42.616635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 393/736] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:42 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:42.621171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-37dec526-353d-4e09-878c-acf995070ed8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:42.623261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-37dec526-353d-4e09-878c-acf995070ed8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:42.628791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37dec526-353d-4e09-878c-acf995070ed8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:42.629271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37dec526-353d-4e09-878c-acf995070ed8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:42.629704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37dec526-353d-4e09-878c-acf995070ed8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:42.630443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37dec526-353d-4e09-878c-acf995070ed8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:42.630894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37dec526-353d-4e09-878c-acf995070ed8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:42.631319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37dec526-353d-4e09-878c-acf995070ed8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:42.659951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-37dec526-353d-4e09-878c-acf995070ed8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042277 Aug 24 10:08:42.660530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 394/737] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:42 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:43.637443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:43.638096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:43.644487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:43.644965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:43.645402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:43.646134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:43.646585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:43.647005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:43.780112 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:43.788451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-83f77708-bdd0-49e3-a8da-a08f019a1a6f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.154817 Aug 24 10:08:43.789032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 395/738] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:43 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:43.793578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-509ff5e6-7713-46da-a2fd-693b52502657 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:43.795658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-509ff5e6-7713-46da-a2fd-693b52502657 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:43.801437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-509ff5e6-7713-46da-a2fd-693b52502657 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:43.802544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-509ff5e6-7713-46da-a2fd-693b52502657 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:43.802982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-509ff5e6-7713-46da-a2fd-693b52502657 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:43.803664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-509ff5e6-7713-46da-a2fd-693b52502657 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:43.804141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-509ff5e6-7713-46da-a2fd-693b52502657 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:43.804567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-509ff5e6-7713-46da-a2fd-693b52502657 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:43.832203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-509ff5e6-7713-46da-a2fd-693b52502657 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042170 Aug 24 10:08:43.832748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 396/739] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:44.809648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:44.810268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:44.816686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:44.817154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:44.818224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:44.818915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:44.819370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:44.819795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:44.949457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:44.957106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-364a6c27-a814-4349-8645-d7b8974ba65d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.151425 Aug 24 10:08:44.957663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 397/740] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:44 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:44.962319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-22dcd14c-861b-4c1b-97a3-9f5a54dce430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:44.964437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-22dcd14c-861b-4c1b-97a3-9f5a54dce430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:44.970013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22dcd14c-861b-4c1b-97a3-9f5a54dce430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:44.970502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22dcd14c-861b-4c1b-97a3-9f5a54dce430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:44.970937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22dcd14c-861b-4c1b-97a3-9f5a54dce430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:44.971627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22dcd14c-861b-4c1b-97a3-9f5a54dce430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:44.972106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22dcd14c-861b-4c1b-97a3-9f5a54dce430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:44.972536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22dcd14c-861b-4c1b-97a3-9f5a54dce430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:45.001618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-22dcd14c-861b-4c1b-97a3-9f5a54dce430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042972 Aug 24 10:08:45.002166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 398/741] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:45.977967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:45.978872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:45.985245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:45.985730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:45.986175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:45.986865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:45.987324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:45.987752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:46.118749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:46.126559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-07c00fb5-5e30-491c-8b2c-0bab535f3a6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.152447 Aug 24 10:08:46.127118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 399/742] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:45 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:46.132147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ee8ef1b4-c70e-4448-8f03-104c6226d56b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:46.134364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ee8ef1b4-c70e-4448-8f03-104c6226d56b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:46.139951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee8ef1b4-c70e-4448-8f03-104c6226d56b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:46.140466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee8ef1b4-c70e-4448-8f03-104c6226d56b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:46.140906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee8ef1b4-c70e-4448-8f03-104c6226d56b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:46.141605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee8ef1b4-c70e-4448-8f03-104c6226d56b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:46.142065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee8ef1b4-c70e-4448-8f03-104c6226d56b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:46.142493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee8ef1b4-c70e-4448-8f03-104c6226d56b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:46.174494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ee8ef1b4-c70e-4448-8f03-104c6226d56b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046331 Aug 24 10:08:46.175068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 400/743] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:46 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:47.148698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:47.149377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:47.156003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:47.156517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:47.156960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:47.157659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:47.158112 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:47.158541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:47.289358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:47.297116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a7c24663-d588-4ac1-9b07-c9a2b329c9d5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.152790 Aug 24 10:08:47.297682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 401/744] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:47 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:47.302155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-97b81ad6-61c5-4b05-9957-0a8961bad45c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:47.304269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-97b81ad6-61c5-4b05-9957-0a8961bad45c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:47.310428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97b81ad6-61c5-4b05-9957-0a8961bad45c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:47.310914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97b81ad6-61c5-4b05-9957-0a8961bad45c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:47.311357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97b81ad6-61c5-4b05-9957-0a8961bad45c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:47.312066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97b81ad6-61c5-4b05-9957-0a8961bad45c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:47.312532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97b81ad6-61c5-4b05-9957-0a8961bad45c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:47.312962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97b81ad6-61c5-4b05-9957-0a8961bad45c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:47.341728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-97b81ad6-61c5-4b05-9957-0a8961bad45c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043009 Aug 24 10:08:47.342367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 402/745] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:48.317083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:48.317732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:48.326487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:48.327259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:48.328011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:48.328848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:48.329545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:48.329979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:48.458422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:48.466228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3efed194-ccfd-4d89-89bd-ca7a380f2c08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.152976 Aug 24 10:08:48.466820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 403/746] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:48 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:48.471330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-985a8812-ef05-47ee-b906-9407539b57be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:48.473443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-985a8812-ef05-47ee-b906-9407539b57be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:48.479812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-985a8812-ef05-47ee-b906-9407539b57be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:48.480320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-985a8812-ef05-47ee-b906-9407539b57be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:48.480756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-985a8812-ef05-47ee-b906-9407539b57be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:48.481440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-985a8812-ef05-47ee-b906-9407539b57be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:48.481891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-985a8812-ef05-47ee-b906-9407539b57be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:48.482315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-985a8812-ef05-47ee-b906-9407539b57be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:48.510974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-985a8812-ef05-47ee-b906-9407539b57be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043147 Aug 24 10:08:48.511533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 404/747] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:48 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:49.487561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:49.488259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:49.494618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:49.495107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:49.495548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:49.496279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:49.496741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:49.497173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:49.628346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:49.630514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:49.637819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-83004e87-403f-47e4-8b5e-f2f71c8ad819 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.154075 Aug 24 10:08:49.638386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 405/748] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:49 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:49.642898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9b5503ec-8c54-43e2-9c1f-f9cf12aa077e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:49.645029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9b5503ec-8c54-43e2-9c1f-f9cf12aa077e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:49.650597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b5503ec-8c54-43e2-9c1f-f9cf12aa077e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:49.651083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b5503ec-8c54-43e2-9c1f-f9cf12aa077e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:49.651525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b5503ec-8c54-43e2-9c1f-f9cf12aa077e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:49.652244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b5503ec-8c54-43e2-9c1f-f9cf12aa077e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:49.652701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b5503ec-8c54-43e2-9c1f-f9cf12aa077e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:49.653131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b5503ec-8c54-43e2-9c1f-f9cf12aa077e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:49.682480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9b5503ec-8c54-43e2-9c1f-f9cf12aa077e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043089 Aug 24 10:08:49.683028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 406/749] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:50.658416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:50.659124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:50.665934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:50.666051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:50.666483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:50.667173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:50.667620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:50.668060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:50.798535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:50.806608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-696bbfac-369b-4b8c-b2a8-0d185a7e0e97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.152052 Aug 24 10:08:50.807158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 407/750] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:50 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:50.811604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4329b064-3883-46d6-8515-ac5ddad0ef3e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:50.813699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4329b064-3883-46d6-8515-ac5ddad0ef3e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:50.819756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4329b064-3883-46d6-8515-ac5ddad0ef3e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:50.820265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4329b064-3883-46d6-8515-ac5ddad0ef3e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:50.820702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4329b064-3883-46d6-8515-ac5ddad0ef3e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:50.821380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4329b064-3883-46d6-8515-ac5ddad0ef3e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:50.821826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4329b064-3883-46d6-8515-ac5ddad0ef3e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:50.822248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4329b064-3883-46d6-8515-ac5ddad0ef3e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:50.852339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4329b064-3883-46d6-8515-ac5ddad0ef3e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044133 Aug 24 10:08:50.852888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 408/751] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:50 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:51.829161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:51.829837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:51.836575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:51.837054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:51.837483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:51.838164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:51.838613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:51.839032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:51.971453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:51.979602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-16ce94de-488d-4c30-bbee-b1262ca09584 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.154639 Aug 24 10:08:51.980189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 409/752] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:51 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:51.984720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e752dfbd-b62d-4f1e-b269-188d88359ebf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:51.986814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e752dfbd-b62d-4f1e-b269-188d88359ebf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:51.992318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e752dfbd-b62d-4f1e-b269-188d88359ebf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:51.992806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e752dfbd-b62d-4f1e-b269-188d88359ebf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:51.993249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e752dfbd-b62d-4f1e-b269-188d88359ebf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:51.993944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e752dfbd-b62d-4f1e-b269-188d88359ebf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:51.994402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e752dfbd-b62d-4f1e-b269-188d88359ebf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:51.994830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e752dfbd-b62d-4f1e-b269-188d88359ebf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:52.025070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e752dfbd-b62d-4f1e-b269-188d88359ebf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043749 Aug 24 10:08:52.025529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 410/753] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:53.000721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:53.001384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:53.007788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:53.008318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:53.008782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:53.009475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:53.009932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:53.010359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:53.142270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:53.150459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f96cded8-7128-4136-a250-f503e0c75fc2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.153563 Aug 24 10:08:53.151024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 411/754] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:52 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:53.155697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a36fa5cc-70bc-4be5-b96d-9226c3e2f161 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:53.158038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a36fa5cc-70bc-4be5-b96d-9226c3e2f161 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:53.164269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a36fa5cc-70bc-4be5-b96d-9226c3e2f161 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:53.164757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a36fa5cc-70bc-4be5-b96d-9226c3e2f161 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:53.165191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a36fa5cc-70bc-4be5-b96d-9226c3e2f161 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:53.165878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a36fa5cc-70bc-4be5-b96d-9226c3e2f161 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:53.166334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a36fa5cc-70bc-4be5-b96d-9226c3e2f161 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:53.166756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a36fa5cc-70bc-4be5-b96d-9226c3e2f161 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:53.195635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a36fa5cc-70bc-4be5-b96d-9226c3e2f161 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043594 Aug 24 10:08:53.196228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 412/755] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:53 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:54.171992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:54.172679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:54.179365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:54.179849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:54.180565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:54.181609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:54.182071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:54.182501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:54.314734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:54.322872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-29988d50-2188-42b9-9c17-4f2516077653 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.154771 Aug 24 10:08:54.323432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 413/756] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:54 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:54.327927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b178f124-ad07-4160-9a66-0412b6e9bbda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:54.330054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b178f124-ad07-4160-9a66-0412b6e9bbda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:54.336192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b178f124-ad07-4160-9a66-0412b6e9bbda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:54.336679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b178f124-ad07-4160-9a66-0412b6e9bbda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:54.337118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b178f124-ad07-4160-9a66-0412b6e9bbda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:54.337815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b178f124-ad07-4160-9a66-0412b6e9bbda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:54.338265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b178f124-ad07-4160-9a66-0412b6e9bbda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:54.338689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b178f124-ad07-4160-9a66-0412b6e9bbda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:54.368505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b178f124-ad07-4160-9a66-0412b6e9bbda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044017 Aug 24 10:08:54.369060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 414/757] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:54 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:55.344694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:55.345393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:55.352394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:55.352883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:55.353576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:55.354269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:55.354730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:55.355160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:55.485346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:55.492970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4e4034c7-fcb9-4635-86d3-6e6be2f9a119 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.152496 Aug 24 10:08:55.493503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 415/758] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:55 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:55.497956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2659076d-18d4-414c-bc56-a585c34bc51e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:55.500015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2659076d-18d4-414c-bc56-a585c34bc51e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:55.505476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2659076d-18d4-414c-bc56-a585c34bc51e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:55.505953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2659076d-18d4-414c-bc56-a585c34bc51e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:55.506384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2659076d-18d4-414c-bc56-a585c34bc51e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:55.507060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2659076d-18d4-414c-bc56-a585c34bc51e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:55.507506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2659076d-18d4-414c-bc56-a585c34bc51e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:55.507923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2659076d-18d4-414c-bc56-a585c34bc51e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:55.536188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2659076d-18d4-414c-bc56-a585c34bc51e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041659 Aug 24 10:08:55.536729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 416/759] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:55 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:56.513983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:56.514646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:56.516916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:56.522966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:56.523451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:56.523891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:56.524608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:56.525078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:56.525500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:56.656514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:56.664241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b8bccacb-38bf-459e-8829-8823f3a90820 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.154120 Aug 24 10:08:56.664805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 417/760] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:56 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:56.669375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-57bb5d48-c9ec-4073-ab9d-86e92d0e06f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:56.671468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-57bb5d48-c9ec-4073-ab9d-86e92d0e06f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:56.677169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57bb5d48-c9ec-4073-ab9d-86e92d0e06f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:56.677653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57bb5d48-c9ec-4073-ab9d-86e92d0e06f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:56.678089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57bb5d48-c9ec-4073-ab9d-86e92d0e06f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:56.678781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57bb5d48-c9ec-4073-ab9d-86e92d0e06f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:56.679243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57bb5d48-c9ec-4073-ab9d-86e92d0e06f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:56.679662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57bb5d48-c9ec-4073-ab9d-86e92d0e06f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:56.707991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-57bb5d48-c9ec-4073-ab9d-86e92d0e06f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042206 Aug 24 10:08:56.708559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 418/761] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:57.684595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:57.685258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:57.691626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:57.692147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:57.692591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:57.693283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:57.693743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:57.694171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:57.825386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:57.833020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-489ad2e0-c8ed-4ee1-b3bc-7617788d8991 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.152255 Aug 24 10:08:57.833580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 419/762] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:57 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:57.838585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4e1e58c8-4663-49bc-8532-3ec291bb93fa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:57.840719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4e1e58c8-4663-49bc-8532-3ec291bb93fa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:57.846347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e1e58c8-4663-49bc-8532-3ec291bb93fa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:57.846833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e1e58c8-4663-49bc-8532-3ec291bb93fa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:57.847272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e1e58c8-4663-49bc-8532-3ec291bb93fa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:57.847958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e1e58c8-4663-49bc-8532-3ec291bb93fa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:57.848513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e1e58c8-4663-49bc-8532-3ec291bb93fa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:57.848940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e1e58c8-4663-49bc-8532-3ec291bb93fa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:57.877775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4e1e58c8-4663-49bc-8532-3ec291bb93fa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043211 Aug 24 10:08:57.878318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 420/763] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:57 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:08:58.855410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:58.856060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:58.862364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:58.862845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:58.863275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:58.863953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:58.864427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:58.864846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:58.994449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:08:59.002144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e88bdb2a-c3e8-4f8f-869c-c7cfbce97abf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.150800 Aug 24 10:08:59.002707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 421/764] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:08:58 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:08:59.007335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f18d068b-1855-4ccd-a7c0-94b15bdddd6f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:08:59.009665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f18d068b-1855-4ccd-a7c0-94b15bdddd6f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:08:59.015269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f18d068b-1855-4ccd-a7c0-94b15bdddd6f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:59.015754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f18d068b-1855-4ccd-a7c0-94b15bdddd6f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:59.016238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f18d068b-1855-4ccd-a7c0-94b15bdddd6f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:59.016926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f18d068b-1855-4ccd-a7c0-94b15bdddd6f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:08:59.017382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f18d068b-1855-4ccd-a7c0-94b15bdddd6f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:08:59.017835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f18d068b-1855-4ccd-a7c0-94b15bdddd6f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:08:59.031548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:59.048153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f18d068b-1855-4ccd-a7c0-94b15bdddd6f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044446 Aug 24 10:08:59.048711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 422/765] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:08:59 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:00.023185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:00.023907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:00.030265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:00.030745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:00.031182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:00.031863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:00.032341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:00.032764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:00.173383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:00.181669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b9512669-a4c3-49ba-9262-56cefe2cd903 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.162570 Aug 24 10:09:00.182249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 423/766] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:00 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:00.186935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1cff0187-4f7b-4765-b366-b9d3cac85555 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:00.189328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1cff0187-4f7b-4765-b366-b9d3cac85555 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:00.196209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cff0187-4f7b-4765-b366-b9d3cac85555 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:00.196746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cff0187-4f7b-4765-b366-b9d3cac85555 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:00.197234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cff0187-4f7b-4765-b366-b9d3cac85555 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:00.197829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cff0187-4f7b-4765-b366-b9d3cac85555 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:00.198284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cff0187-4f7b-4765-b366-b9d3cac85555 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:00.198710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cff0187-4f7b-4765-b366-b9d3cac85555 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:00.229780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1cff0187-4f7b-4765-b366-b9d3cac85555 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046472 Aug 24 10:09:00.230342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 424/767] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:00 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:01.084258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:01.084946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:01.091294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:01.091782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:01.092264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:01.092962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:01.093418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:01.093845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:01.119213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:09:01.194734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-51eda450-d598-472f-941b-9132169dd9c4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 204 len: 0 microversion: 2.1 time: 0.114157 Aug 24 10:09:01.195523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 425/768] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:09:01 2022] DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 24 10:09:01.204357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:01.205053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:01.210767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:01.211248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:01.211680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:01.212397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:01.212845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:01.213265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:01.344837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:01.353126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-43bbaa9c-31a9-4e02-9ae2-5f74148b461a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.153510 Aug 24 10:09:01.353935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 426/769] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:01 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:01.358272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:01.358868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:01.364519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:01.365015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:01.365496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:01.366193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:01.366654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:01.367151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:01.495037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:01.502716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e46d6db7-9539-4d06-b190-b619c7fddabb tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1576 microversion: 2.1 time: 0.147854 Aug 24 10:09:01.503280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 427/770] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:01 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1576 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:01.507789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fb7dcb2f-301b-4093-8581-c4831c1306f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:01.509925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fb7dcb2f-301b-4093-8581-c4831c1306f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:01.515430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb7dcb2f-301b-4093-8581-c4831c1306f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:01.515917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb7dcb2f-301b-4093-8581-c4831c1306f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:01.516396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb7dcb2f-301b-4093-8581-c4831c1306f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:01.517114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb7dcb2f-301b-4093-8581-c4831c1306f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:01.517602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb7dcb2f-301b-4093-8581-c4831c1306f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:01.518031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb7dcb2f-301b-4093-8581-c4831c1306f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:01.549806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fb7dcb2f-301b-4093-8581-c4831c1306f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045512 Aug 24 10:09:01.550350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 428/771] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:02.374126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:02.374798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:02.381816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:02.382329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:02.382773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:02.383497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:02.384072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:02.384500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:02.517710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:02.525381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5eccca58-0d52-482c-8035-570e585e312a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.155118 Aug 24 10:09:02.525972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 429/772] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:02 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:02.530192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:02.530793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:02.536262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:02.536752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:02.537868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:02.538577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:02.539040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:02.539475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:02.673998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:02.682427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-21f4dcf9-3035-4337-bbcd-e330b0a121f8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1576 microversion: 2.1 time: 0.155485 Aug 24 10:09:02.682983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 430/773] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:02 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1576 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:02.687780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d2494ee8-3de2-4d39-92a3-ef36952392b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:02.689882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d2494ee8-3de2-4d39-92a3-ef36952392b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:02.695735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2494ee8-3de2-4d39-92a3-ef36952392b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:02.696270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2494ee8-3de2-4d39-92a3-ef36952392b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:02.696855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2494ee8-3de2-4d39-92a3-ef36952392b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:02.697519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2494ee8-3de2-4d39-92a3-ef36952392b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:02.697972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2494ee8-3de2-4d39-92a3-ef36952392b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:02.698392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2494ee8-3de2-4d39-92a3-ef36952392b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:02.726940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d2494ee8-3de2-4d39-92a3-ef36952392b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042710 Aug 24 10:09:02.727487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 431/774] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:02 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:03.546618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:03.547307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:03.554091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:03.554674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:03.555115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:03.556093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:03.556288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:03.556714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:03.808900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:03.816862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-255c708c-074a-40d7-8b0e-ff7c1cf582f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.274055 Aug 24 10:09:03.817434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 432/775] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:03 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:03.822484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:03.823096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:03.828924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:03.829486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:03.830231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:03.830574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:03.831090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:03.831529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:03.953802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:03.962741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-febf7824-0c3f-41b7-b627-1d91b15caaa8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1509 microversion: 2.1 time: 0.144263 Aug 24 10:09:03.968541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 433/776] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:03 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1509 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:03.976925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f9750f96-632e-4c29-bb65-1f6b378417b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:03.979425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f9750f96-632e-4c29-bb65-1f6b378417b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:03.985872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9750f96-632e-4c29-bb65-1f6b378417b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:03.986368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9750f96-632e-4c29-bb65-1f6b378417b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:03.986808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9750f96-632e-4c29-bb65-1f6b378417b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:03.987513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9750f96-632e-4c29-bb65-1f6b378417b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:03.988145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9750f96-632e-4c29-bb65-1f6b378417b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:03.988664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f9750f96-632e-4c29-bb65-1f6b378417b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:04.025122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f9750f96-632e-4c29-bb65-1f6b378417b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053846 Aug 24 10:09:04.025651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 434/777] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:04.112927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-19b791f8-a841-484e-9a49-5b8e074299bb req-6c753a02-de95-4136-8132-0558498e0981 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "350e1445-9336-439c-b146-ee4c74211b54", "name": "network-vif-deleted", "tag": "e7af2444-9e35-49f8-aeab-b94839769f32"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:09:04.122881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-19b791f8-a841-484e-9a49-5b8e074299bb req-6c753a02-de95-4136-8132-0558498e0981 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:04.123443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-19b791f8-a841-484e-9a49-5b8e074299bb req-6c753a02-de95-4136-8132-0558498e0981 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:04.123943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-19b791f8-a841-484e-9a49-5b8e074299bb req-6c753a02-de95-4136-8132-0558498e0981 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:04.135722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-19b791f8-a841-484e-9a49-5b8e074299bb req-6c753a02-de95-4136-8132-0558498e0981 service nova] Creating event network-vif-deleted:e7af2444-9e35-49f8-aeab-b94839769f32 for instance 350e1445-9336-439c-b146-ee4c74211b54 on hv-cinder-85427 Aug 24 10:09:04.140978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-19b791f8-a841-484e-9a49-5b8e074299bb req-6c753a02-de95-4136-8132-0558498e0981 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089871 Aug 24 10:09:04.141588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 435/778] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:09:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:09:04.839984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:04.840718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:04.847731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:04.848355 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:04.848867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:04.849640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:04.850163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:04.850657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:04.989236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:04.997653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0893a4c0-2f19-4791-9935-bd2874c798ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.161847 Aug 24 10:09:04.998288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 436/779] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:05.002999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:05.003612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:05.011213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:05.011698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:05.012161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:05.012967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:05.013403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:05.013827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:05.082045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:05.090264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d36c2edf-6510-4df9-a859-7f1f8326584f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.090922 Aug 24 10:09:05.090839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 437/780] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:04 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:05.095445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-30b912e1-ccf6-4efb-a000-712e01d07085 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:05.097672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-30b912e1-ccf6-4efb-a000-712e01d07085 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:05.104212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b912e1-ccf6-4efb-a000-712e01d07085 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:05.104705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b912e1-ccf6-4efb-a000-712e01d07085 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:05.105145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b912e1-ccf6-4efb-a000-712e01d07085 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:05.105842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b912e1-ccf6-4efb-a000-712e01d07085 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:05.106302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b912e1-ccf6-4efb-a000-712e01d07085 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:05.106760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b912e1-ccf6-4efb-a000-712e01d07085 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:05.109406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:05.139079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-30b912e1-ccf6-4efb-a000-712e01d07085 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047206 Aug 24 10:09:05.139659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 438/781] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:06.019443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:06.020100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:06.026325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:06.026762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:06.027209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:06.027905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:06.028391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:06.028822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:06.368404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:06.378192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-63c88530-c83e-4c0a-b86a-8102a3976e0c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.362651 Aug 24 10:09:06.378875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 439/782] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:06 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:06.386088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:06.387060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:06.393207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:06.393848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:06.394313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:06.395017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:06.395473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:06.395906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:06.460794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:06.469119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3a184d21-ba58-45b6-80c2-087d59898fa5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.088964 Aug 24 10:09:06.469721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 440/783] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:06 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:06.475345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a9d6c71e-9160-490c-b0f4-6c0200c3de98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:06.477580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a9d6c71e-9160-490c-b0f4-6c0200c3de98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:06.484176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9d6c71e-9160-490c-b0f4-6c0200c3de98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:06.484676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9d6c71e-9160-490c-b0f4-6c0200c3de98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:06.485122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9d6c71e-9160-490c-b0f4-6c0200c3de98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:06.485840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9d6c71e-9160-490c-b0f4-6c0200c3de98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:06.486337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9d6c71e-9160-490c-b0f4-6c0200c3de98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:06.486767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9d6c71e-9160-490c-b0f4-6c0200c3de98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:06.518991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a9d6c71e-9160-490c-b0f4-6c0200c3de98 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048111 Aug 24 10:09:06.519575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 441/784] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:07.399047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:07.399655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:07.406155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:07.406642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:07.407083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:07.407775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:07.408269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:07.408698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:07.539965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:07.547786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9a2d55dd-a755-4fa7-9c41-bb6a12fea71a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.152704 Aug 24 10:09:07.548385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 442/785] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:07 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:07.553347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:07.553982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:07.559525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:07.560018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:07.560492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:07.561188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:07.561644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:07.562075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:07.622664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:07.630326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a673ab69-c1e0-4db5-a7e3-dbbb23faf7a5 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.080940 Aug 24 10:09:07.630892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 443/786] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:07 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:07.636133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-22fb312b-594a-478b-b6b0-2218eca3756f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:07.638287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-22fb312b-594a-478b-b6b0-2218eca3756f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:07.643874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22fb312b-594a-478b-b6b0-2218eca3756f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:07.644400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22fb312b-594a-478b-b6b0-2218eca3756f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:07.644841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22fb312b-594a-478b-b6b0-2218eca3756f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:07.645539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22fb312b-594a-478b-b6b0-2218eca3756f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:07.646005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22fb312b-594a-478b-b6b0-2218eca3756f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:07.646468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22fb312b-594a-478b-b6b0-2218eca3756f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:07.675348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-22fb312b-594a-478b-b6b0-2218eca3756f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042838 Aug 24 10:09:07.675826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 444/787] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:08.570460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:08.571097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:08.577478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:08.577961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:08.578418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:08.579106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:08.579562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:08.579988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:08.707916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:08.716376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-25317a7c-2f2c-40ec-979f-252a5b512ee2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.150394 Aug 24 10:09:08.716936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 445/788] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:08 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:08.721261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:08.721862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:08.727520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:08.728010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:08.728481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:08.729426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:08.729882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:08.730309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:08.793856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:08.801481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0c188aae-0c8a-4f97-a698-f7f6b480a7ab tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.083629 Aug 24 10:09:08.802035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 446/789] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:08 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:08.807104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-29fb7068-bb40-4b66-a088-1e6f6346f690 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:08.809210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-29fb7068-bb40-4b66-a088-1e6f6346f690 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:08.814648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29fb7068-bb40-4b66-a088-1e6f6346f690 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:08.815121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29fb7068-bb40-4b66-a088-1e6f6346f690 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:08.815551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29fb7068-bb40-4b66-a088-1e6f6346f690 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:08.816261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29fb7068-bb40-4b66-a088-1e6f6346f690 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:08.816714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29fb7068-bb40-4b66-a088-1e6f6346f690 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:08.817134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-29fb7068-bb40-4b66-a088-1e6f6346f690 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:08.845729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-29fb7068-bb40-4b66-a088-1e6f6346f690 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042648 Aug 24 10:09:08.846191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 447/790] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:08 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:09.737998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:09.738625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:09.745527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:09.746016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:09.746460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:09.747152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:09.747611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:09.748070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:09.877075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:09.885364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a9a7134d-2259-4d13-a605-49d8432274ce tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.151291 Aug 24 10:09:09.885967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 448/791] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:09 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:09.890420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:09.891063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:09.896573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:09.897059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:09.897502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:09.898235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:09.898697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:09.899122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:09.959985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:09.967934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1af301d3-cc15-4fd7-a3c0-f473511dea6a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.081026 Aug 24 10:09:09.968562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 449/792] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:09 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:09.974760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-73af8c9c-a6a8-4d21-93c8-e2de5f7045e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:09.977583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-73af8c9c-a6a8-4d21-93c8-e2de5f7045e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:09.983859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73af8c9c-a6a8-4d21-93c8-e2de5f7045e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:09.984370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73af8c9c-a6a8-4d21-93c8-e2de5f7045e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:09.984866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73af8c9c-a6a8-4d21-93c8-e2de5f7045e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:09.985591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73af8c9c-a6a8-4d21-93c8-e2de5f7045e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:09.986107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73af8c9c-a6a8-4d21-93c8-e2de5f7045e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:09.986559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73af8c9c-a6a8-4d21-93c8-e2de5f7045e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:10.017294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-73af8c9c-a6a8-4d21-93c8-e2de5f7045e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046372 Aug 24 10:09:10.017846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 450/793] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:09 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:10.906551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:10.907213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:10.913649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:10.914143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:10.914581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:10.915286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:10.915740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:10.916193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:11.140359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:11.149017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cf3564af-67c2-4ee5-9643-23d9d73c912c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.246250 Aug 24 10:09:11.149615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 451/794] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:10 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:11.154550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:11.155168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:11.161274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:11.161753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:11.162194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:11.162896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:11.163358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:11.164367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:11.262505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:11.270319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-702c5be4-9034-4205-9fac-3ce26a6f182f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.119690 Aug 24 10:09:11.270892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 452/795] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:11 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:11.276635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-10515c88-78bd-4381-bc62-48b08b366845 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:11.278747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-10515c88-78bd-4381-bc62-48b08b366845 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:11.284889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10515c88-78bd-4381-bc62-48b08b366845 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:11.285363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10515c88-78bd-4381-bc62-48b08b366845 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:11.285801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10515c88-78bd-4381-bc62-48b08b366845 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:11.286506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10515c88-78bd-4381-bc62-48b08b366845 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:11.286956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10515c88-78bd-4381-bc62-48b08b366845 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:11.287381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10515c88-78bd-4381-bc62-48b08b366845 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:11.318292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-10515c88-78bd-4381-bc62-48b08b366845 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046320 Aug 24 10:09:11.319173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 453/796] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:11 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:12.174100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:12.174739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:12.181013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:12.181497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:12.181933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:12.182616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:12.183071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:12.183498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:12.327217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:12.329452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:12.337076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-137334ad-e923-4c1f-b01b-60a46ddb7dd6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.167037 Aug 24 10:09:12.337655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 454/797] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:12 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:12.342320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:12.342924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:12.348905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:12.349393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:12.349835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:12.350531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:12.351064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:12.351510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:12.417080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:12.425030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2e48cd6e-c442-458f-b594-0d5c15ba5ac8 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.086356 Aug 24 10:09:12.425527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 455/798] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:12 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:12.430122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f519bc7c-bcc5-43be-8e7e-ef8e6177a736 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:12.432296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f519bc7c-bcc5-43be-8e7e-ef8e6177a736 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:12.437982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f519bc7c-bcc5-43be-8e7e-ef8e6177a736 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:12.438460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f519bc7c-bcc5-43be-8e7e-ef8e6177a736 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:12.438895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f519bc7c-bcc5-43be-8e7e-ef8e6177a736 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:12.439584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f519bc7c-bcc5-43be-8e7e-ef8e6177a736 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:12.440067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f519bc7c-bcc5-43be-8e7e-ef8e6177a736 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:12.440488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f519bc7c-bcc5-43be-8e7e-ef8e6177a736 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:12.469603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f519bc7c-bcc5-43be-8e7e-ef8e6177a736 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043092 Aug 24 10:09:12.470147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 456/799] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:12 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:13.358691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:13.359375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:13.365897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:13.366381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:13.366829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:13.367514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:13.367969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:13.368422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:13.496849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:13.504545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-64451437-4ee0-42bc-93f2-75c385157b38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.149722 Aug 24 10:09:13.505125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 457/800] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:13 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:13.509687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:13.510292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:13.515775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:13.516316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:13.516757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:13.517448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:13.517902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:13.518327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:13.578368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:13.586657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ff40f076-37fc-4b63-95f9-5e5185017295 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.080439 Aug 24 10:09:13.587124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 458/801] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:13 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:13.591849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5ea8bd9a-b72a-41f1-a63e-5a5076e3b57f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:13.594039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5ea8bd9a-b72a-41f1-a63e-5a5076e3b57f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:13.599576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ea8bd9a-b72a-41f1-a63e-5a5076e3b57f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:13.600086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ea8bd9a-b72a-41f1-a63e-5a5076e3b57f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:13.600534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ea8bd9a-b72a-41f1-a63e-5a5076e3b57f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:13.601792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ea8bd9a-b72a-41f1-a63e-5a5076e3b57f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:13.602252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ea8bd9a-b72a-41f1-a63e-5a5076e3b57f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:13.602721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ea8bd9a-b72a-41f1-a63e-5a5076e3b57f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:13.634542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5ea8bd9a-b72a-41f1-a63e-5a5076e3b57f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046309 Aug 24 10:09:13.635025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 459/802] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:14.525810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:14.526494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:14.532891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:14.533378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:14.533819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:14.534512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:14.534970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:14.535405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:14.670622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:14.678429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-132c68ca-5934-4aa7-a89d-575f29b71c13 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.157125 Aug 24 10:09:14.679029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 460/803] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:14 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:14.683434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:14.684068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:14.689706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:14.690189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:14.690629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:14.691316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:14.691774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:14.692255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:14.756895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:14.765425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7f50142c-e14d-42da-b057-c17a9bcc4d60 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.085444 Aug 24 10:09:14.765993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 461/804] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:14 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:14.771179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-291fb269-3c2f-4b33-ace4-0344ebf6f840 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:14.772812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-291fb269-3c2f-4b33-ace4-0344ebf6f840 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:14.778792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-291fb269-3c2f-4b33-ace4-0344ebf6f840 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:14.779216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-291fb269-3c2f-4b33-ace4-0344ebf6f840 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:14.779656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-291fb269-3c2f-4b33-ace4-0344ebf6f840 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:14.780381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-291fb269-3c2f-4b33-ace4-0344ebf6f840 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:14.780845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-291fb269-3c2f-4b33-ace4-0344ebf6f840 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:14.781273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-291fb269-3c2f-4b33-ace4-0344ebf6f840 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:14.811376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-291fb269-3c2f-4b33-ace4-0344ebf6f840 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044353 Aug 24 10:09:14.811927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 462/805] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:14 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:15.699482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:15.700204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:15.706583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:15.707061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:15.707492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:15.708208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:15.708654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:15.709075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:15.839462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:15.847177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e0e53e37-44c5-4012-9d5d-2bbc74236815 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.151898 Aug 24 10:09:15.847732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 463/806] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:15 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:15.852636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:15.853234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:15.855069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:15.860227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:15.860722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:15.861142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:15.861823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:15.862275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:15.862693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:15.927195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:15.934859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-33d01c63-e42c-4d48-93b3-94a73e2b90f7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.086175 Aug 24 10:09:15.935412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 464/807] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:15 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:15.944319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-955a1aee-4f51-49f3-ad42-4a97cd07fa25 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:15.944319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-955a1aee-4f51-49f3-ad42-4a97cd07fa25 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:15.948167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-955a1aee-4f51-49f3-ad42-4a97cd07fa25 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:15.948658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-955a1aee-4f51-49f3-ad42-4a97cd07fa25 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:15.949099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-955a1aee-4f51-49f3-ad42-4a97cd07fa25 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:15.949871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-955a1aee-4f51-49f3-ad42-4a97cd07fa25 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:15.950280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-955a1aee-4f51-49f3-ad42-4a97cd07fa25 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:15.950708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-955a1aee-4f51-49f3-ad42-4a97cd07fa25 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:15.980072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-955a1aee-4f51-49f3-ad42-4a97cd07fa25 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043634 Aug 24 10:09:15.980623 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 465/808] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:16.869416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:16.870072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:16.876383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:16.876864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:16.877294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:16.877969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:16.878416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:16.878836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:17.006808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:17.015128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cb57f325-f418-47e4-bf98-278a6cd466a7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.149590 Aug 24 10:09:17.015688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 466/809] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:16 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:17.020030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:17.020651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:17.026247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:17.026731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:17.027163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:17.027845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:17.028331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:17.028745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:17.089515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:17.097242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f05fd3a5-5965-4a4b-87ed-c50230f32c48 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.080598 Aug 24 10:09:17.097803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 467/810] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:17 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:17.102301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-37a95eb7-343f-4e21-95a2-0469a4fe0da2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:17.104668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-37a95eb7-343f-4e21-95a2-0469a4fe0da2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:17.110338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37a95eb7-343f-4e21-95a2-0469a4fe0da2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:17.110826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37a95eb7-343f-4e21-95a2-0469a4fe0da2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:17.111273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37a95eb7-343f-4e21-95a2-0469a4fe0da2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:17.111980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37a95eb7-343f-4e21-95a2-0469a4fe0da2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:17.112460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37a95eb7-343f-4e21-95a2-0469a4fe0da2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:17.112889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-37a95eb7-343f-4e21-95a2-0469a4fe0da2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:17.142392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-37a95eb7-343f-4e21-95a2-0469a4fe0da2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043571 Aug 24 10:09:17.142942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 468/811] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:18.037290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:18.037911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:18.045067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:18.045588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:18.046309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:18.047482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:18.048320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:18.049072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:18.184688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:18.192297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-097809c8-34bd-4bae-a918-d23e2e65f666 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.158989 Aug 24 10:09:18.192855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 469/812] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:18 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:18.197674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:18.198277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:18.203717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:18.204239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:18.204681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:18.205371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:18.205830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:18.206257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:18.266302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:18.274494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a5488c3b-780d-46de-97c4-ff5d17991912 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.080703 Aug 24 10:09:18.275053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 470/813] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:18 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:18.279540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8da7fc9a-9545-4437-a34d-f2f1c5ef22de tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:18.281654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8da7fc9a-9545-4437-a34d-f2f1c5ef22de tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:18.287139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8da7fc9a-9545-4437-a34d-f2f1c5ef22de tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:18.287670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8da7fc9a-9545-4437-a34d-f2f1c5ef22de tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:18.288078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8da7fc9a-9545-4437-a34d-f2f1c5ef22de tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:18.288849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8da7fc9a-9545-4437-a34d-f2f1c5ef22de tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:18.289300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8da7fc9a-9545-4437-a34d-f2f1c5ef22de tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:18.289725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8da7fc9a-9545-4437-a34d-f2f1c5ef22de tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:18.303582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:18.319016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8da7fc9a-9545-4437-a34d-f2f1c5ef22de tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042994 Aug 24 10:09:18.319564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 471/814] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:18 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:19.213625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:19.214293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:19.220617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:19.221109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:19.221549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:19.222228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:19.222702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:19.223126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:19.350044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:19.357981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-529e61a6-89e6-4b1b-8467-c4497dca60a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.148812 Aug 24 10:09:19.358537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 472/815] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:19 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:19.362827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:19.363426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:19.369330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:19.369808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:19.370247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:19.370937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:19.371390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:19.371813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:19.435013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:19.442984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1da1d864-b0d6-4d13-b048-d511d4d9b07d tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.083513 Aug 24 10:09:19.443542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 473/816] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:19 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:19.448975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-73b70621-0d0f-4714-8c0f-89baaa472c7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:19.452027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-73b70621-0d0f-4714-8c0f-89baaa472c7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:19.459314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73b70621-0d0f-4714-8c0f-89baaa472c7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:19.459800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73b70621-0d0f-4714-8c0f-89baaa472c7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:19.460270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73b70621-0d0f-4714-8c0f-89baaa472c7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:19.460988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73b70621-0d0f-4714-8c0f-89baaa472c7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:19.461448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73b70621-0d0f-4714-8c0f-89baaa472c7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:19.462350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73b70621-0d0f-4714-8c0f-89baaa472c7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:19.491322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-73b70621-0d0f-4714-8c0f-89baaa472c7d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046709 Aug 24 10:09:19.492323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 474/817] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:20.379662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:20.380377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:20.387233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:20.387713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:20.388182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:20.388862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:20.389321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:20.389744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:20.518754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:20.526421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1cb090e1-d74e-4f27-a96e-90de97c7559c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.150609 Aug 24 10:09:20.526976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 475/818] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:20 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:20.531375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:20.531981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:20.537442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:20.537927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:20.538362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:20.539583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:20.540038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:20.540483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:20.599695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:20.608513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-33279593-2462-4afa-939e-332581976fb4 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.080547 Aug 24 10:09:20.609073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 476/819] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:20 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:20.613665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e9656841-de60-4064-9dd8-081e2563ca8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:20.615747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e9656841-de60-4064-9dd8-081e2563ca8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:20.621959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9656841-de60-4064-9dd8-081e2563ca8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:20.622419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9656841-de60-4064-9dd8-081e2563ca8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:20.622867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9656841-de60-4064-9dd8-081e2563ca8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:20.623534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9656841-de60-4064-9dd8-081e2563ca8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:20.623984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9656841-de60-4064-9dd8-081e2563ca8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:20.624430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9656841-de60-4064-9dd8-081e2563ca8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:20.653053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e9656841-de60-4064-9dd8-081e2563ca8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042901 Aug 24 10:09:20.653517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 477/820] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:20 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:21.547751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:21.548439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:21.555309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:21.555792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:21.556362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:21.557050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:21.557509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:21.557937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:21.690253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:21.698624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2a544a91-97d5-4462-b7fa-2358287aef26 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.154717 Aug 24 10:09:21.699190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 478/821] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:21 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:09:21.703585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:21.704351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:21.710213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:21.710951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:21.711424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:21.712147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:21.712608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:21.713045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:21.773832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:09:21.781760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-819b4171-8c39-45ba-99d7-dc757826b74b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.081635 Aug 24 10:09:21.782342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 479/822] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:09:21 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:09:21.787093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0526b267-461c-45ea-887e-15e50eb4965b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:21.789245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0526b267-461c-45ea-887e-15e50eb4965b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:21.795521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0526b267-461c-45ea-887e-15e50eb4965b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:21.796002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0526b267-461c-45ea-887e-15e50eb4965b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:21.796469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0526b267-461c-45ea-887e-15e50eb4965b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:21.797165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0526b267-461c-45ea-887e-15e50eb4965b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:21.797618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0526b267-461c-45ea-887e-15e50eb4965b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:21.798091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0526b267-461c-45ea-887e-15e50eb4965b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:21.827925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0526b267-461c-45ea-887e-15e50eb4965b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044556 Aug 24 10:09:21.828505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 480/823] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:09:21 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:09:21.832997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:09:21.833491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:09:21.839250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:21.839730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:21.840197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:21.840885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:09:21.841340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:09:21.841768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:09:31.837086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=35 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:31.859615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=61 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:03.381580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:10:03.494207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:10:13.424000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:13.498889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:15.206787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:11:15.207472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:11:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:11:26.934319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:11:26.934872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:11:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:11:30.612268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:11:30.612999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:11:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:11:32.046915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:11:32.047406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:11:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:11:44.220680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:11:44.221418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:11:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:12:03.680265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b429f958-bd2d-4a87-935e-6b1476f87f67 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 202 len: 0 microversion: 2.1 time: 221.944272 Aug 24 10:12:03.680907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 344/824] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:08:21 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 221946 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:12:03.688033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:12:03.688915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:12:03.697256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:03.697753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:03.698566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:03.699747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:03.700505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:03.700715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:03.703420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-df9add50-648c-413b-b6d0-b240882ba1d4) Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:12:03.705874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:12:03.708211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-df9add50-648c-413b-b6d0-b240882ba1d4) Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:12:03.709445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-df9add50-648c-413b-b6d0-b240882ba1d4) Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:12:03.714170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-df9add50-648c-413b-b6d0-b240882ba1d4) Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:12:03.717647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-df9add50-648c-413b-b6d0-b240882ba1d4) Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:12:03.720914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-df9add50-648c-413b-b6d0-b240882ba1d4) Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-df9add50-648c-413b-b6d0-b240882ba1d4) Aug 24 10:12:03.725636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:12:03.728114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-df9add50-648c-413b-b6d0-b240882ba1d4) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:12:03.728114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d0c77d2b-9675-4df8-a6ab-a37fee525cd6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 161.878169 Aug 24 10:12:03.728114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 481/825] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:09:21 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 269 bytes in 161880 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:12:03.861363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:12:03.872522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fd45d77a-2923-403d-af3f-16db9ba369c0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.189882 Aug 24 10:12:03.873348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 345/826] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:12:03 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1568 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:12:03.879453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-baa8feab-f335-400d-8a4f-802644395230 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:12:03.880791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-baa8feab-f335-400d-8a4f-802644395230 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:12:03.889293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baa8feab-f335-400d-8a4f-802644395230 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:03.890030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baa8feab-f335-400d-8a4f-802644395230 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:03.890902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baa8feab-f335-400d-8a4f-802644395230 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:03.892181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baa8feab-f335-400d-8a4f-802644395230 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:03.892321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baa8feab-f335-400d-8a4f-802644395230 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:03.894168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-baa8feab-f335-400d-8a4f-802644395230 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:03.902104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:12:03.902885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:12:03.910613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:03.911120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:03.911560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:03.912292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:03.912914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:03.913356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:03.942723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-baa8feab-f335-400d-8a4f-802644395230 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068097 Aug 24 10:12:03.943749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 346/827] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:12:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:12:03.951918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:12:03.951918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:12:03.958117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:03.958117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:03.958117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:03.958795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:03.959402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:03.959996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:03.989794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:12:03.998981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-98f02830-1398-431e-9782-6fe9d7ca97bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.289114 Aug 24 10:12:04.001774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 482/828] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:12:03 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 1225 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:12:04.004702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:12:04.005057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:12:04.011189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:04.011675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:04.012175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:04.012874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:12:04.013407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:12:04.013834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:12:13.956380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=6 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:13.982066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=17,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:14.009205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:14.033513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=10 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:04.078087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:14:04.166815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:14:14.082194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:14.170622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:35.217123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:14:35.217819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:14:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:14:46.943595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:14:46.944151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:14:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:14:50.620838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:14:50.621614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:14:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:14:52.054804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:14:52.055334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:14:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:15:04.230288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:15:04.230982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:15:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:15:08.720898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:15:08.721476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:15:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:16:04.464561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 202 len: 0 microversion: 2.1 time: 240.518973 Aug 24 10:16:04.465502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:16:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:16:04.465502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:16:04 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:16:04.465809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-185b59bc-ffde-46b4-a1ec-3dc91c77216d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:16:04.465809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:16:04.465809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:16:04.466142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 347/829] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:12:03 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 240522 msecs (HTTP/1.1 202) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:16:04.471628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:16:04.472505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:16:04.479680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.480441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.481244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:04.482569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.483173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.483945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:04.487708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-29b0da11-d8f9-4004-a15e-77355b824185) Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:16:04.492819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:16:04.494592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-29b0da11-d8f9-4004-a15e-77355b824185) Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:16:04.495327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-29b0da11-d8f9-4004-a15e-77355b824185) Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:16:04.496497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-29b0da11-d8f9-4004-a15e-77355b824185) Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:16:04.497816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-29b0da11-d8f9-4004-a15e-77355b824185) Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:16:04.498980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-29b0da11-d8f9-4004-a15e-77355b824185) Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-29b0da11-d8f9-4004-a15e-77355b824185) Aug 24 10:16:04.502473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:16:04.509689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-29b0da11-d8f9-4004-a15e-77355b824185) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:16:04.509689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 240.505010 Aug 24 10:16:04.512697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:16:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:16:04.513019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:16:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (10.222.0.86) Aug 24 10:16:04.515124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-f298f5f4-9838-4e62-9019-34931805a486 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Unhandled error: OSError: write error Aug 24 10:16:04.515124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:16:04.515124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:16:04.515799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 483/830] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:12:03 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 240516 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:16:04.527791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:16:04.529550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:16:04.540180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.540934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.541681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:04.542622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.543306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.543896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:04.622777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:16:04.629140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:16:04.631540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.113931 Aug 24 10:16:04.632078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:16:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (ip 10.222.0.86) !!! Aug 24 10:16:04.632078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:16:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (10.222.0.86) Aug 24 10:16:04.632672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-5fb4898f-ff34-4ee4-aa0e-661fe36d346a tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Unhandled error: OSError: write error Aug 24 10:16:04.632672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:16:04.632672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:16:04.632936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 484/831] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:16:04 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:16:04.637414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.170304 Aug 24 10:16:04.638017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:16:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (ip 10.222.0.86) !!! Aug 24 10:16:04.638017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:16:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (10.222.0.86) Aug 24 10:16:04.638676 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-45136657-5982-4d14-b76f-ba32984e3daa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Unhandled error: OSError: write error Aug 24 10:16:04.638676 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:16:04.638676 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:16:04.638923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 348/832] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:16:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:16:04.643410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:16:04.643886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:16:04.650675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.651165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.651599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:04.652328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.652777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.653200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:04.710834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:16:04.713334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:16:04.719596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.720348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.720795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:04.721556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.722068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.722660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:04.756065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.122110 Aug 24 10:16:04.756592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:16:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:16:04.756592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:16:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (10.222.0.86) Aug 24 10:16:04.757165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-fdff63e4-a866-4029-8aab-2fe0aaf3b93e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Unhandled error: OSError: write error Aug 24 10:16:04.757165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:16:04.757165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:16:04.757538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 485/833] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:16:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 0 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:16:04.762122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:16:04.762619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:16:04.768936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.769432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.769973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:04.770561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:16:04.771015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:16:04.771439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:16:14.648021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:14.673073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=11,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:14.765833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=6 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:14.790544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=16 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:55.225424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:17:55.226202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:17:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:18:04.775311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:18:04.883466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:18:06.953045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:18:06.953479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:18:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:18:10.629296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:18:10.629961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:18:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:18:12.062409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:18:12.062966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:18:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:18:14.779848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:14.887390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:24.238215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:18:24.238818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:18:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:18:28.729262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:18:28.729836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:18:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:20:05.254418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8afcaa1f-dc98-4aa5-85b6-2ffcfabfa16e) Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:20:05.256699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:20:05.258363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8afcaa1f-dc98-4aa5-85b6-2ffcfabfa16e) Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:20:05.258999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8afcaa1f-dc98-4aa5-85b6-2ffcfabfa16e) Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:20:05.260500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8afcaa1f-dc98-4aa5-85b6-2ffcfabfa16e) Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.262449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8afcaa1f-dc98-4aa5-85b6-2ffcfabfa16e) Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:20:05.263767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8afcaa1f-dc98-4aa5-85b6-2ffcfabfa16e) Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8afcaa1f-dc98-4aa5-85b6-2ffcfabfa16e) Aug 24 10:20:05.265149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.266641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8afcaa1f-dc98-4aa5-85b6-2ffcfabfa16e) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:20:05.266641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 240.618421 Aug 24 10:20:05.266641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:20:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:20:05.266641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:20:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:20:05.266641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-b4e3c888-b0a8-47b5-a130-0c9b361ba61f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:20:05.266641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:20:05.266641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:20:05.267417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 349/834] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:16:04 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 240620 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:20:05.267417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:20:05.267417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:20:05.271288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.271777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.272264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:05.272948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.273931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.274345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:05.341857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-85d985df-04b9-4a13-ba55-30a1f97a5e01) Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:20:05.344846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:20:05.346646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-85d985df-04b9-4a13-ba55-30a1f97a5e01) Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:20:05.347296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-85d985df-04b9-4a13-ba55-30a1f97a5e01) Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:20:05.349103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-85d985df-04b9-4a13-ba55-30a1f97a5e01) Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.353011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-85d985df-04b9-4a13-ba55-30a1f97a5e01) Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:20:05.355258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-85d985df-04b9-4a13-ba55-30a1f97a5e01) Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-85d985df-04b9-4a13-ba55-30a1f97a5e01) Aug 24 10:20:05.358663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:20:05.361021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-85d985df-04b9-4a13-ba55-30a1f97a5e01) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:20:05.361021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 240.587560 Aug 24 10:20:05.361021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:20:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:20:05.361021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:20:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (10.222.0.86) Aug 24 10:20:05.361021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-bc15985c-2988-4264-ace0-ec7c9e872ee8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Unhandled error: OSError: write error Aug 24 10:20:05.361021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:20:05.361021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:20:05.362018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 486/835] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:16:04 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 240589 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:20:05.362018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:20:05.362018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:20:05.364030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.364822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.365477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:05.365957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.366419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.366880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:05.413352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:20:05.422125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1568 microversion: 2.1 time: 0.161234 Aug 24 10:20:05.422651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:20:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (ip 10.222.0.86) !!! Aug 24 10:20:05.422651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:20:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (10.222.0.86) Aug 24 10:20:05.423239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-cf2d0eb7-b7b0-439b-b4e0-e01035180e06 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Unhandled error: OSError: write error Aug 24 10:20:05.423239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:20:05.423239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:20:05.423569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 350/836] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:20:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:20:05.427919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:20:05.430214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:20:05.433450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:20:05.435995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.436484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.436925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:05.437620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.438076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.438503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:05.441450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1225 microversion: 2.1 time: 0.093254 Aug 24 10:20:05.441962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:20:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (ip 10.222.0.86) !!! Aug 24 10:20:05.441962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:20:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (10.222.0.86) Aug 24 10:20:05.442545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-8e57fef1-1454-453f-8392-39328dc15230 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Unhandled error: OSError: write error Aug 24 10:20:05.442545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:20:05.442545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:20:05.442793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 487/837] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:20:05 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:20:05.447297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:20:05.447792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:20:05.453495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.453980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.454421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:05.455118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.455577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.456008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:05.468353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043702 Aug 24 10:20:05.468860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:20:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:20:05.468860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:20:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (10.222.0.86) Aug 24 10:20:05.469435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-9faae731-789b-4b86-84d3-c6439f8aa0bf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Unhandled error: OSError: write error Aug 24 10:20:05.469435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:20:05.469435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:20:05.469713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 351/838] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:20:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 0 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:20:05.474040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:20:05.474531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:20:05.480863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.481348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.481807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:05.482482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:20:05.482936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:20:05.483364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:20:15.451032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:15.474351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=10 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:15.478746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=6 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:15.501017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=16 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:21:15.233765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:21:15.234386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:21:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:21:26.963136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:21:26.963729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:21:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:21:30.637501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:21:30.638167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:21:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:21:32.070340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:21:32.070850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:21:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:21:44.247192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:21:44.247862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:21:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:21:48.738313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:21:48.738957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:21:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:22:05.607768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:22:05.648673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:22:15.612128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=4 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:15.653200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:04.114657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a2e3fcfb-631d-40be-89db-8ecc3583e5d5) Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:04.116961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:04.118385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a2e3fcfb-631d-40be-89db-8ecc3583e5d5) Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.119037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a2e3fcfb-631d-40be-89db-8ecc3583e5d5) Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.120226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a2e3fcfb-631d-40be-89db-8ecc3583e5d5) Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.121943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a2e3fcfb-631d-40be-89db-8ecc3583e5d5) Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.123050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a2e3fcfb-631d-40be-89db-8ecc3583e5d5) Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a2e3fcfb-631d-40be-89db-8ecc3583e5d5) Aug 24 10:24:04.124372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.127602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a2e3fcfb-631d-40be-89db-8ecc3583e5d5) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:04.127602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 238.675051 Aug 24 10:24:04.127602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:24:04.127602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:24:04.127602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-6796e901-9678-4b44-b910-b3622650d980 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:24:04.127602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:04.127602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:04.128259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 488/839] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:20:05 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 238677 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.261933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0eb58ff-687b-4cb1-a246-3593b65ac4f0) Aug 24 10:24:04.270745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.271362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:04.274695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:04.276463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0eb58ff-687b-4cb1-a246-3593b65ac4f0) Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.277217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0eb58ff-687b-4cb1-a246-3593b65ac4f0) Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.278295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0eb58ff-687b-4cb1-a246-3593b65ac4f0) Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.279502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0eb58ff-687b-4cb1-a246-3593b65ac4f0) Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.283197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0eb58ff-687b-4cb1-a246-3593b65ac4f0) Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0eb58ff-687b-4cb1-a246-3593b65ac4f0) Aug 24 10:24:04.285392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.287141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.287141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.287141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.287141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.291928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.291928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.292236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0eb58ff-687b-4cb1-a246-3593b65ac4f0) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:04.292236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 238.818637 Aug 24 10:24:04.292236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:24:04.292236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (10.222.0.86) Aug 24 10:24:04.292236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-a3cb3120-4a2c-493b-9f23-6d63cd75931d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Unhandled error: OSError: write error Aug 24 10:24:04.292236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:04.292236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:04.292864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 352/840] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:20:05 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 238820 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.435763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:04.445213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1562 microversion: 2.1 time: 0.322949 Aug 24 10:24:04.445802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (ip 10.222.0.86) !!! Aug 24 10:24:04.445802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (10.222.0.86) Aug 24 10:24:04.446392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-109e1cf1-7cd3-4811-adcd-6ec35137c411 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Unhandled error: OSError: write error Aug 24 10:24:04.446392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:04.446392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:04.446649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 489/841] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.447188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.447806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.451287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.454159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.454542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.455037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.455477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.456206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.456669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.457094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.460606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.461300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.461909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.462778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.463435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.463865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.495527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.496014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.496505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.496953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lazy-loading 'fault' on Instance uuid 350e1445-9336-439c-b146-ee4c74211b54 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:04.500572 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052876 Aug 24 10:24:04.501082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:24:04.501082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (10.222.0.86) Aug 24 10:24:04.501664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-805dc7d2-21c6-48cc-910f-c5fc6e79a418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Unhandled error: OSError: write error Aug 24 10:24:04.501664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:04.501664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:04.501937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 490/842] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:24:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 0 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.507542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.508036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.513925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.514433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.514848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.516119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.516579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.517008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.545182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ccfd2f7e35646b0a60b28b6807cee4b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc33f347b062491aaa04b7dc18553cf4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:04.554929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 200 len: 1230 microversion: 2.1 time: 0.263268 Aug 24 10:24:04.555820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (ip 10.222.0.86) !!! Aug 24 10:24:04.555820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (10.222.0.86) Aug 24 10:24:04.556464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-eecf23bc-6493-45d0-bccf-0b2540d7664c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Unhandled error: OSError: write error Aug 24 10:24:04.556464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:04.556464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:04.556921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 353/843] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:04 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.561928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.562996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.568922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.569444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.570044 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.571180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.571681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.572340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.613489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:04.653358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:04.680986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-751d1b32-c546-4091-8e5a-3b3eccf0d122) Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:04.683148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:04.684727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-751d1b32-c546-4091-8e5a-3b3eccf0d122) Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.685595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-751d1b32-c546-4091-8e5a-3b3eccf0d122) Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.686657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-751d1b32-c546-4091-8e5a-3b3eccf0d122) Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.688454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-751d1b32-c546-4091-8e5a-3b3eccf0d122) Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.689486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-751d1b32-c546-4091-8e5a-3b3eccf0d122) Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-751d1b32-c546-4091-8e5a-3b3eccf0d122) Aug 24 10:24:04.690593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.691674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-751d1b32-c546-4091-8e5a-3b3eccf0d122) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:04.691674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 0.181895 Aug 24 10:24:04.691674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:24:04.691674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:24:04.691674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-93ebf74a-3c98-4908-b03f-c73e125c6768 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:24:04.691674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:04.691674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:04.693674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 491/844] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:04 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 183 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.693674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.693674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.697150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.697626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.698062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.698743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.699259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.699717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.716614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cc44ec5e-5634-417f-81a2-36306a66cfcb) Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:04.718804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:04.720497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cc44ec5e-5634-417f-81a2-36306a66cfcb) Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.721279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cc44ec5e-5634-417f-81a2-36306a66cfcb) Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.722579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cc44ec5e-5634-417f-81a2-36306a66cfcb) Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.724062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cc44ec5e-5634-417f-81a2-36306a66cfcb) Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.726580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cc44ec5e-5634-417f-81a2-36306a66cfcb) Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cc44ec5e-5634-417f-81a2-36306a66cfcb) Aug 24 10:24:04.727953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.730301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-cc44ec5e-5634-417f-81a2-36306a66cfcb) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:04.730301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.162321 Aug 24 10:24:04.730301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:24:04.730301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (10.222.0.86) Aug 24 10:24:04.730301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-9c39afe3-e5e0-4cb3-ad3c-167c33dde0f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Unhandled error: OSError: write error Aug 24 10:24:04.730301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:04.730301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:04.730906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 354/845] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:04 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 164 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.730906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.730906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.733731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.734764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.735270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.736035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.736576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.737056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.751346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:04.753006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:04.754402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:04.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:04.756500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.757669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:04.757669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.032229 Aug 24 10:24:04.757669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (ip 10.222.0.86) !!! Aug 24 10:24:04.757669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (10.222.0.86) Aug 24 10:24:04.757669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-90aa6fc9-17f4-47cd-9451-128c5a30791c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Unhandled error: OSError: write error Aug 24 10:24:04.757669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:04.757669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:04.757669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 355/846] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:04 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.762500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.765154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.771413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.771738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.772191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.772935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.773365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.773745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.807715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050819 Aug 24 10:24:04.808177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:24:04.808177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (10.222.0.86) Aug 24 10:24:04.808756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-747c503f-4c1d-48fe-9516-4bb5e70747da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Unhandled error: OSError: write error Aug 24 10:24:04.808756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:04.808756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:04.809062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 356/847] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:24:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 0 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.814245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.814421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.821565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.822049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.822477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.823171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.823620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.824036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.864863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:04.873003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1562 microversion: 2.1 time: 0.186417 Aug 24 10:24:04.873509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (ip 10.222.0.86) !!! Aug 24 10:24:04.873509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (10.222.0.86) Aug 24 10:24:04.874113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-6d422e8c-eec8-4c19-a735-e12f84da31e2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Unhandled error: OSError: write error Aug 24 10:24:04.874113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:04.874113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:04.874381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 492/848] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.906462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:04.964523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2be7dbe0-7834-4aa7-87c0-222d3591a0a5) Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:04.966712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:04.968162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2be7dbe0-7834-4aa7-87c0-222d3591a0a5) Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.969307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2be7dbe0-7834-4aa7-87c0-222d3591a0a5) Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:04.970423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2be7dbe0-7834-4aa7-87c0-222d3591a0a5) Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.971569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2be7dbe0-7834-4aa7-87c0-222d3591a0a5) Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.973727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2be7dbe0-7834-4aa7-87c0-222d3591a0a5) Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2be7dbe0-7834-4aa7-87c0-222d3591a0a5) Aug 24 10:24:04.975252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:04.976759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2be7dbe0-7834-4aa7-87c0-222d3591a0a5) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:04.976759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 0.158340 Aug 24 10:24:04.976759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:24:04.976759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:04 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:24:04.976759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-fbcbae5d-dc8f-4570-9ce2-3879432f0f09 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:24:04.976759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:04.976759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:04.977378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 357/849] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:04 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 160 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:04.977378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:04.977378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:04.981126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.981609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.982041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:04.982726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:04.983180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:04.983598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.016562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.017068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.022962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.023459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.024071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.024620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.025076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.025504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.066126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:05.107581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:05.135937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6122831b-09ff-4db5-b72c-e65e4d4c37f5) Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:05.138204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:05.139792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6122831b-09ff-4db5-b72c-e65e4d4c37f5) Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6122831b-09ff-4db5-b72c-e65e4d4c37f5) Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.142104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6122831b-09ff-4db5-b72c-e65e4d4c37f5) Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.143497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6122831b-09ff-4db5-b72c-e65e4d4c37f5) Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.144817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6122831b-09ff-4db5-b72c-e65e4d4c37f5) Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6122831b-09ff-4db5-b72c-e65e4d4c37f5) Aug 24 10:24:05.146115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.147417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6122831b-09ff-4db5-b72c-e65e4d4c37f5) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:05.147417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.168923 Aug 24 10:24:05.147417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:24:05.147417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (10.222.0.86) Aug 24 10:24:05.147417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-ef8dc31f-2274-4275-8b01-b3b3c0b7ec17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Unhandled error: OSError: write error Aug 24 10:24:05.147417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:05.147417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:05.147964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 358/850] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:04 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 170 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.147964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.147964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.151684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.152202 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.152633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.153338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.153784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.154200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.197025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 202 len: 0 microversion: 2.1 time: 0.320729 Aug 24 10:24:05.197622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (ip 10.222.0.86) !!! Aug 24 10:24:05.197622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (10.222.0.86) Aug 24 10:24:05.198203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-65872b2c-d7e5-4867-ae68-6f379af75f27 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Unhandled error: OSError: write error Aug 24 10:24:05.198203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:05.198203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:05.198490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 493/851] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:04 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 0 bytes in 324 msecs (HTTP/1.1 202) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.203849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.204575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.213713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.214467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.215060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.216021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.216823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.217214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.236603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:05.238158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.239512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:05.240357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.241711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.242873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:05.242873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.040179 Aug 24 10:24:05.242873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (ip 10.222.0.86) !!! Aug 24 10:24:05.242873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (10.222.0.86) Aug 24 10:24:05.242873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-ebfc276d-0f30-4d31-afee-5fc7d0ced3b6 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Unhandled error: OSError: write error Aug 24 10:24:05.242873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:05.242873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:05.242873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 494/852] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:05 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.245454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.248402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.255002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.255614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.256086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.256851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.257304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.257732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.288960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046989 Aug 24 10:24:05.289491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:24:05.289491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (10.222.0.86) Aug 24 10:24:05.290071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-1f885274-c440-4eaa-9510-1b89dd270f92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Unhandled error: OSError: write error Aug 24 10:24:05.290071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:05.290071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:05.290300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 495/853] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:24:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 0 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.294877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.295360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.297571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:05.302082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.302604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.303507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.303761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.304246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.304671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.308452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1562 microversion: 2.1 time: 0.166481 Aug 24 10:24:05.308920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (ip 10.222.0.86) !!! Aug 24 10:24:05.308920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (10.222.0.86) Aug 24 10:24:05.309762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-a5d70d6b-b9c3-4623-a31d-b8e9656d0b23 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Unhandled error: OSError: write error Aug 24 10:24:05.309762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:05.309762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:05.310057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 359/854] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.315787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.316316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.322757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.323491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.324151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.325053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.325970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.326614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.387919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:05.448507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-acc1c69d-ba29-4c77-8515-9e1a5d8736fc) Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:05.450887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:05.452307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-acc1c69d-ba29-4c77-8515-9e1a5d8736fc) Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.453634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-acc1c69d-ba29-4c77-8515-9e1a5d8736fc) Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.454680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-acc1c69d-ba29-4c77-8515-9e1a5d8736fc) Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.455846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-acc1c69d-ba29-4c77-8515-9e1a5d8736fc) Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.456969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-acc1c69d-ba29-4c77-8515-9e1a5d8736fc) Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-acc1c69d-ba29-4c77-8515-9e1a5d8736fc) Aug 24 10:24:05.458160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.459496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-acc1c69d-ba29-4c77-8515-9e1a5d8736fc) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:05.459496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 0.161234 Aug 24 10:24:05.459496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:24:05.459496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:24:05.459496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-2ba782ed-78fb-45e5-85a6-82eaf40e8280 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:24:05.459496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:05.459496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:05.460014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 496/855] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:05 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 163 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.463158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.463158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.467958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.469056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.469489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.470189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.470635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.471061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.552361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:05.594064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:05.620878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f7c90d7-ad28-4d47-9a9c-8ee31a931147) Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:05.623039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:05.624621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f7c90d7-ad28-4d47-9a9c-8ee31a931147) Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.625678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f7c90d7-ad28-4d47-9a9c-8ee31a931147) Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.626940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f7c90d7-ad28-4d47-9a9c-8ee31a931147) Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.628438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f7c90d7-ad28-4d47-9a9c-8ee31a931147) Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.629799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f7c90d7-ad28-4d47-9a9c-8ee31a931147) Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f7c90d7-ad28-4d47-9a9c-8ee31a931147) Aug 24 10:24:05.631111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.632401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f7c90d7-ad28-4d47-9a9c-8ee31a931147) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:05.632401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.169175 Aug 24 10:24:05.632401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:24:05.632401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (10.222.0.86) Aug 24 10:24:05.632401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-dee84731-8b84-4a68-9149-81a434858a0c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Unhandled error: OSError: write error Aug 24 10:24:05.632401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:05.632401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:05.632957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 497/856] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:05 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 171 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.632957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.632957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.636851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.637344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.637791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.638515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.638975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.639403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.664221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9cfb02cd-ea36-41bf-88c6-f232d22fe59c) Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:05.665138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:05.666538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9cfb02cd-ea36-41bf-88c6-f232d22fe59c) Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.667175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9cfb02cd-ea36-41bf-88c6-f232d22fe59c) Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.668242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9cfb02cd-ea36-41bf-88c6-f232d22fe59c) Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.669403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9cfb02cd-ea36-41bf-88c6-f232d22fe59c) Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.670450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9cfb02cd-ea36-41bf-88c6-f232d22fe59c) Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9cfb02cd-ea36-41bf-88c6-f232d22fe59c) Aug 24 10:24:05.671584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.672650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9cfb02cd-ea36-41bf-88c6-f232d22fe59c) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:05.672650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.355686 Aug 24 10:24:05.672650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (ip 10.222.0.86) !!! Aug 24 10:24:05.672650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (10.222.0.86) Aug 24 10:24:05.672650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-93dd3861-9a18-4963-9f13-61dabb8513c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Unhandled error: OSError: write error Aug 24 10:24:05.672650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:05.672650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:05.673450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 360/857] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:05 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 0 bytes in 357 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.673450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.673450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.679156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.679645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.680101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.680831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.681282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.681701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.697536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:05.700149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.701661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:05.702405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:05.704648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.706129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:05.706129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.033159 Aug 24 10:24:05.706129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (ip 10.222.0.86) !!! Aug 24 10:24:05.706129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (10.222.0.86) Aug 24 10:24:05.706129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-fcf2caa1-0e72-4786-8355-ac0d89c448fe tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Unhandled error: OSError: write error Aug 24 10:24:05.706129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:05.706129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:05.706129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 361/858] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:05 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.712470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.715686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.725011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.725513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.726007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.726679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.727139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.727565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.758992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053240 Aug 24 10:24:05.759491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:24:05.759491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (10.222.0.86) Aug 24 10:24:05.760089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-990f67c2-d7a1-43b5-8772-acb42dc8b471 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Unhandled error: OSError: write error Aug 24 10:24:05.760089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:05.760089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:05.760428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 362/859] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:24:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 0 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.764711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.765538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.772547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.773035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.773885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.774033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:05.774154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.774592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.774991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.781888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1562 microversion: 2.1 time: 0.155397 Aug 24 10:24:05.782427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (ip 10.222.0.86) !!! Aug 24 10:24:05.782427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (10.222.0.86) Aug 24 10:24:05.785190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-a4707da9-cd3a-4446-a076-cd30e64cde31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Unhandled error: OSError: write error Aug 24 10:24:05.785190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:05.785190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:05.785475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 498/860] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.791044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.791540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.797492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.797983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.798460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.799154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.799650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.800105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.857574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:05.900690 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:05.928265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-68552212-90f8-4462-ad83-922a39f44a06) Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:05.930823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:05.932311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-68552212-90f8-4462-ad83-922a39f44a06) Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.933060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-68552212-90f8-4462-ad83-922a39f44a06) Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.934155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-68552212-90f8-4462-ad83-922a39f44a06) Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.936426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-68552212-90f8-4462-ad83-922a39f44a06) Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.940111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-68552212-90f8-4462-ad83-922a39f44a06) Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-68552212-90f8-4462-ad83-922a39f44a06) Aug 24 10:24:05.941671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.943081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-68552212-90f8-4462-ad83-922a39f44a06) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:05.943081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 0.171203 Aug 24 10:24:05.943081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:24:05.943081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:24:05.943081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-12364d3b-a534-47c4-bf6c-a0c23f269260 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:24:05.943081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:05.943081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:05.943672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 363/861] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:05 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 173 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.943672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.943672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.945456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.945943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.946381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.947080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.947534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.947955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.968589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b813a38c-e549-491a-911d-4536856fb3f7) Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:05.970796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:05.972370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b813a38c-e549-491a-911d-4536856fb3f7) Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.973000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b813a38c-e549-491a-911d-4536856fb3f7) Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:05.974033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b813a38c-e549-491a-911d-4536856fb3f7) Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.975178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b813a38c-e549-491a-911d-4536856fb3f7) Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.976289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b813a38c-e549-491a-911d-4536856fb3f7) Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b813a38c-e549-491a-911d-4536856fb3f7) Aug 24 10:24:05.977847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:05.979078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b813a38c-e549-491a-911d-4536856fb3f7) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:05.979078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.185940 Aug 24 10:24:05.979078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (ip 10.222.0.86) !!! Aug 24 10:24:05.979078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:05 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (10.222.0.86) Aug 24 10:24:05.979078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-1dd2b315-84c4-4cf3-8351-44a5cbc300be tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Unhandled error: OSError: write error Aug 24 10:24:05.979078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:05.979078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:05.979622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 499/862] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:05 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 0 bytes in 188 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:05.979622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:05.979622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:05.984765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.985251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.985682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:05.986371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:05.986824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:05.987243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.032338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:06.091199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd12f347-29de-4db2-a5fd-c2a3314ce43b) Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:06.093569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:06.095182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd12f347-29de-4db2-a5fd-c2a3314ce43b) Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.095922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd12f347-29de-4db2-a5fd-c2a3314ce43b) Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.097539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd12f347-29de-4db2-a5fd-c2a3314ce43b) Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.098942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd12f347-29de-4db2-a5fd-c2a3314ce43b) Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.100198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd12f347-29de-4db2-a5fd-c2a3314ce43b) Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd12f347-29de-4db2-a5fd-c2a3314ce43b) Aug 24 10:24:06.102535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.104908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd12f347-29de-4db2-a5fd-c2a3314ce43b) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:06.104908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.160480 Aug 24 10:24:06.104908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:24:06.104908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (10.222.0.86) Aug 24 10:24:06.104908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-8d7e9d48-68e8-45f1-92c5-a16496513480 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Unhandled error: OSError: write error Aug 24 10:24:06.104908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:06.104908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:06.105511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 364/863] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:05 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 162 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.105511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.105511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.108279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.109550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.110001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.110799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.111190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.111614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.128895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:06.130456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.132176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:06.133124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.134630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.135991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:06.135991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.034783 Aug 24 10:24:06.135991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (ip 10.222.0.86) !!! Aug 24 10:24:06.135991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (10.222.0.86) Aug 24 10:24:06.135991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-6b53766d-a5dd-4a72-bf94-8a461695ba0b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Unhandled error: OSError: write error Aug 24 10:24:06.135991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:06.135991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:06.135991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 365/864] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:06 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.137985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.140660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.146912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.147414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.147835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.148573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.149032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.149460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.182478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048139 Aug 24 10:24:06.183010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:24:06.183010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (10.222.0.86) Aug 24 10:24:06.183596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-adfed9bd-9479-4595-8933-ef550be24b13 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Unhandled error: OSError: write error Aug 24 10:24:06.183596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:06.183596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:06.183883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 366/865] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:24:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 0 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.189386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.189880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.195970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.196547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.196994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.197698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.198570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.199008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.281629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:06.342029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:06.342536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2ca93718-72b1-48d0-8eee-9f79364915fc) Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:06.344701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:06.346226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2ca93718-72b1-48d0-8eee-9f79364915fc) Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.346968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2ca93718-72b1-48d0-8eee-9f79364915fc) Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.348233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2ca93718-72b1-48d0-8eee-9f79364915fc) Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.349646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2ca93718-72b1-48d0-8eee-9f79364915fc) Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.351051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2ca93718-72b1-48d0-8eee-9f79364915fc) Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2ca93718-72b1-48d0-8eee-9f79364915fc) Aug 24 10:24:06.352792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.355073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2ca93718-72b1-48d0-8eee-9f79364915fc) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:06.355073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 0.161463 Aug 24 10:24:06.355073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:24:06.355073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:24:06.355073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-01deebcb-d757-4f27-a0b7-054be4ac0780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:24:06.355073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:06.355073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:06.355856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 367/866] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:06 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 163 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.355956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1562 microversion: 2.1 time: 0.375457 Aug 24 10:24:06.355956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (ip 10.222.0.86) !!! Aug 24 10:24:06.355956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (10.222.0.86) Aug 24 10:24:06.355956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-6acbffdf-78f4-4868-bdf9-cadbf026aa69 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Unhandled error: OSError: write error Aug 24 10:24:06.355956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:06.355956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:06.355956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 500/867] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.356642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.356642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.356829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.356829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.359580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.360157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.360745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.361448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.361919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.362329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.363081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.363568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.364002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.364726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.365185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.365634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.439593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:06.444289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:06.484633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:06.512514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5dedec8a-2094-4cf2-b3ce-39a177178c90) Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:06.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:06.516340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5dedec8a-2094-4cf2-b3ce-39a177178c90) Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.517223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5dedec8a-2094-4cf2-b3ce-39a177178c90) Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.518280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5dedec8a-2094-4cf2-b3ce-39a177178c90) Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.519503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5dedec8a-2094-4cf2-b3ce-39a177178c90) Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.520588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5dedec8a-2094-4cf2-b3ce-39a177178c90) Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5dedec8a-2094-4cf2-b3ce-39a177178c90) Aug 24 10:24:06.521684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.522736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5dedec8a-2094-4cf2-b3ce-39a177178c90) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:06.522736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.168006 Aug 24 10:24:06.522736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (ip 10.222.0.86) !!! Aug 24 10:24:06.522736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (10.222.0.86) Aug 24 10:24:06.522736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-f8c911dc-fc10-42fd-99d2-c44ce625975d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Unhandled error: OSError: write error Aug 24 10:24:06.522736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:06.522736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:06.523280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 368/868] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:06 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 0 bytes in 170 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.523280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.523280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.528939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.529418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.529848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.530533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.531501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.531923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.548519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f90438e-4869-4a5b-9aa9-aa17351a1772) Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:06.551041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:06.552428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f90438e-4869-4a5b-9aa9-aa17351a1772) Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.553286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f90438e-4869-4a5b-9aa9-aa17351a1772) Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.554418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f90438e-4869-4a5b-9aa9-aa17351a1772) Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.555585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f90438e-4869-4a5b-9aa9-aa17351a1772) Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.558211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f90438e-4869-4a5b-9aa9-aa17351a1772) Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f90438e-4869-4a5b-9aa9-aa17351a1772) Aug 24 10:24:06.564491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.567372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5f90438e-4869-4a5b-9aa9-aa17351a1772) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:06.567372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.200647 Aug 24 10:24:06.567372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:24:06.567372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (10.222.0.86) Aug 24 10:24:06.567372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-730b90d6-7172-4955-a233-e5f0bdf92d3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Unhandled error: OSError: write error Aug 24 10:24:06.567372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:06.567372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:06.568092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 501/869] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:06 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 202 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.568092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.568092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.570233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.570742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.571154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.571855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.572625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.573057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.589307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:06.590934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.592916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:06.594847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:06.596697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.598111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:06.598111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.037497 Aug 24 10:24:06.598111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (ip 10.222.0.86) !!! Aug 24 10:24:06.598111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (10.222.0.86) Aug 24 10:24:06.598111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-fd2caecc-33e1-4ec1-b579-a16a12ba582f tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Unhandled error: OSError: write error Aug 24 10:24:06.598111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:06.598111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:06.598111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 502/870] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:06 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.602597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.605209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.611717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.612382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.612889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.613649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.614448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.615302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.645395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050047 Aug 24 10:24:06.645905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:24:06.645905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (10.222.0.86) Aug 24 10:24:06.646488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-eca67b00-96a3-4ca4-8964-ce19194f7e42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Unhandled error: OSError: write error Aug 24 10:24:06.646488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:06.646488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:06.646745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 503/871] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:24:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 0 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.651134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.651628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.657688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.658205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.658646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.659340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.659799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.660265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.669079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:06.677514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1562 microversion: 2.1 time: 0.159297 Aug 24 10:24:06.678024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (ip 10.222.0.86) !!! Aug 24 10:24:06.678024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (10.222.0.86) Aug 24 10:24:06.678604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-ca99446b-921f-4c41-a012-9cc42e1f20fe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Unhandled error: OSError: write error Aug 24 10:24:06.678604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:06.678604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:06.678898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:06.678997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 369/872] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:06 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.683761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.684285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.689971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.690454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.691081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.691665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.692142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.692572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.742338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:06.787284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:06.816391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-52ba2404-7c1a-4b96-823d-bd41174b4571) Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:06.818647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:06.820182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-52ba2404-7c1a-4b96-823d-bd41174b4571) Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.821212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-52ba2404-7c1a-4b96-823d-bd41174b4571) Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.822369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-52ba2404-7c1a-4b96-823d-bd41174b4571) Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.823555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-52ba2404-7c1a-4b96-823d-bd41174b4571) Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.825274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-52ba2404-7c1a-4b96-823d-bd41174b4571) Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-52ba2404-7c1a-4b96-823d-bd41174b4571) Aug 24 10:24:06.826889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.829105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-52ba2404-7c1a-4b96-823d-bd41174b4571) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:06.829105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 0.174130 Aug 24 10:24:06.829105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:24:06.829105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:24:06.829105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-6f60fbee-6c45-4cfc-b654-3e16d9e9ff9f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:24:06.829105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:06.829105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:06.829670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 504/873] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:06 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 176 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.829670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.829670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.835045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.835644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.836136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.836860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.837316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.837742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.851675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eaa614bc-1bab-4a7d-868d-1e5b534133f5) Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:06.853949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:06.855563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eaa614bc-1bab-4a7d-868d-1e5b534133f5) Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.856240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eaa614bc-1bab-4a7d-868d-1e5b534133f5) Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.857512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eaa614bc-1bab-4a7d-868d-1e5b534133f5) Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.858891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eaa614bc-1bab-4a7d-868d-1e5b534133f5) Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.860240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eaa614bc-1bab-4a7d-868d-1e5b534133f5) Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eaa614bc-1bab-4a7d-868d-1e5b534133f5) Aug 24 10:24:06.861605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.862888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eaa614bc-1bab-4a7d-868d-1e5b534133f5) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:06.862888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.175480 Aug 24 10:24:06.862888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (ip 10.222.0.86) !!! Aug 24 10:24:06.862888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (10.222.0.86) Aug 24 10:24:06.862888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-37cf12ce-ba5d-4c91-ab3f-f9573af8020e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Unhandled error: OSError: write error Aug 24 10:24:06.862888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:06.862888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:06.863477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 370/874] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:06 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 0 bytes in 177 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.863477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.863477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.868028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.868547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.869062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.869757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.870201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.870613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.918152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:06.974589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd43b009-282e-42c8-a67d-898226c43e15) Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:06.976876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:06.978414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd43b009-282e-42c8-a67d-898226c43e15) Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.979061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd43b009-282e-42c8-a67d-898226c43e15) Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:06.980415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd43b009-282e-42c8-a67d-898226c43e15) Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.981675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd43b009-282e-42c8-a67d-898226c43e15) Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.983487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd43b009-282e-42c8-a67d-898226c43e15) Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd43b009-282e-42c8-a67d-898226c43e15) Aug 24 10:24:06.984730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:06.986240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-dd43b009-282e-42c8-a67d-898226c43e15) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:06.986240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.154402 Aug 24 10:24:06.986240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (ip 10.222.0.86) !!! Aug 24 10:24:06.986240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:06 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e (10.222.0.86) Aug 24 10:24:06.986240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-4001ac3f-8d6c-488a-bd13-eef514de1278 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Unhandled error: OSError: write error Aug 24 10:24:06.986240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:06.986240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:06.987354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 505/875] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:06 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 0 bytes in 156 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:06.987354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:06.987354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:06.991225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.991713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.992199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:06.992892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:06.993368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:06.993786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.004514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:07.007369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:07.008933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.010500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:07.011311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.012597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.013756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:07.013756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.029955 Aug 24 10:24:07.013756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (ip 10.222.0.86) !!! Aug 24 10:24:07.013756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:07 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 (10.222.0.86) Aug 24 10:24:07.013756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-2b85da1b-f3fa-4632-8045-21f929a1350c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Unhandled error: OSError: write error Aug 24 10:24:07.013756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:07.013756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:07.013756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 506/876] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:06 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 0 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:07.014290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1562 microversion: 2.1 time: 0.154713 Aug 24 10:24:07.014290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (ip 10.222.0.86) !!! Aug 24 10:24:07.014290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:07 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 (10.222.0.86) Aug 24 10:24:07.014290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-eca9ae46-7a3c-41c4-acbb-6d4c5819a292 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Unhandled error: OSError: write error Aug 24 10:24:07.014290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:07.014290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:07.014290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 371/877] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:06 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:07.016540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.018650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.018861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.019181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.024430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.024782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.024905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.025265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.025411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.026005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.026333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.026483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.026866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.027026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.027489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.027912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.057612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045216 Aug 24 10:24:07.058706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:24:07.058706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:07 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments (10.222.0.86) Aug 24 10:24:07.058706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-07d26da3-e72d-4196-81e6-ece7768a75ab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Unhandled error: OSError: write error Aug 24 10:24:07.058706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:07.058706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:07.059166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 507/878] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:24:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 0 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:07.063881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.064393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.070327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.071165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.071613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.072383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.072844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.073269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.111017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:07.173076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d9e9c91-7303-4936-9ba8-291dbf5a9132) Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:07.175260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:07.176727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d9e9c91-7303-4936-9ba8-291dbf5a9132) Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.177444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d9e9c91-7303-4936-9ba8-291dbf5a9132) Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.178487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d9e9c91-7303-4936-9ba8-291dbf5a9132) Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.179764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d9e9c91-7303-4936-9ba8-291dbf5a9132) Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.180910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d9e9c91-7303-4936-9ba8-291dbf5a9132) Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d9e9c91-7303-4936-9ba8-291dbf5a9132) Aug 24 10:24:07.182239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.183435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d9e9c91-7303-4936-9ba8-291dbf5a9132) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:07.183435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 0.162390 Aug 24 10:24:07.184513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (ip 10.222.0.86) !!! Aug 24 10:24:07.184513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:07 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac (10.222.0.86) Aug 24 10:24:07.190010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: CRITICAL nova [None req-e6e83618-b2d8-49ea-9a02-dd05517da62d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Unhandled error: OSError: write error Aug 24 10:24:07.190010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova OSError: write error Aug 24 10:24:07.190010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova Aug 24 10:24:07.190345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 372/879] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 0 bytes in 176 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:07.196697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.197383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.205779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.206274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.206711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.207455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.207907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.208384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.290481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:07.333830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:07.362357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b92b9e3b-0166-4cc3-8b0e-f50bf9daf6bd) Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:07.364561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:07.366119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b92b9e3b-0166-4cc3-8b0e-f50bf9daf6bd) Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.367187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b92b9e3b-0166-4cc3-8b0e-f50bf9daf6bd) Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.368320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b92b9e3b-0166-4cc3-8b0e-f50bf9daf6bd) Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.369479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b92b9e3b-0166-4cc3-8b0e-f50bf9daf6bd) Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.370507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b92b9e3b-0166-4cc3-8b0e-f50bf9daf6bd) Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b92b9e3b-0166-4cc3-8b0e-f50bf9daf6bd) Aug 24 10:24:07.371654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.372872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b92b9e3b-0166-4cc3-8b0e-f50bf9daf6bd) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:07.372872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-65aa7323-9deb-4654-830a-a338abfdb7e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.174823 Aug 24 10:24:07.372872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 373/880] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 269 bytes in 176 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:07.372872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.373316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.378612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.379090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.379522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.380241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.380693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.381111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.397998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d4e26a9-72b3-42e6-8ce1-c7f20e298f0a) Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:07.400196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:07.401857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d4e26a9-72b3-42e6-8ce1-c7f20e298f0a) Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d4e26a9-72b3-42e6-8ce1-c7f20e298f0a) Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.403877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d4e26a9-72b3-42e6-8ce1-c7f20e298f0a) Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.405549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d4e26a9-72b3-42e6-8ce1-c7f20e298f0a) Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.407182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d4e26a9-72b3-42e6-8ce1-c7f20e298f0a) Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d4e26a9-72b3-42e6-8ce1-c7f20e298f0a) Aug 24 10:24:07.408638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.410100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0d4e26a9-72b3-42e6-8ce1-c7f20e298f0a) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:07.410100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.341925 Aug 24 10:24:07.410100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (ip 10.222.0.86) !!! Aug 24 10:24:07.410100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:07 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f (10.222.0.86) Aug 24 10:24:07.410100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: CRITICAL nova [None req-4c840957-0f75-4b26-afe6-ea17359c45a4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Unhandled error: OSError: write error Aug 24 10:24:07.410100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova OSError: write error Aug 24 10:24:07.410100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova Aug 24 10:24:07.410774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 508/881] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 0 bytes in 344 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Aug 24 10:24:07.410774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.410774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.415195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.415693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.416159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.416857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.417311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.417736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.431371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:07.432968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.434609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:07.435490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.437855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.439510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:07.439510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5569fe39-b0ae-4129-b3ea-5632079405a7 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.029659 Aug 24 10:24:07.439510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 509/882] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:07 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:24:07.440291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eb1c7da0-c652-4c31-80c7-78fdb7ab120f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.442460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eb1c7da0-c652-4c31-80c7-78fdb7ab120f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.448101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb1c7da0-c652-4c31-80c7-78fdb7ab120f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.448586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb1c7da0-c652-4c31-80c7-78fdb7ab120f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.449025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb1c7da0-c652-4c31-80c7-78fdb7ab120f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.449717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb1c7da0-c652-4c31-80c7-78fdb7ab120f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.450173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb1c7da0-c652-4c31-80c7-78fdb7ab120f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.450600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb1c7da0-c652-4c31-80c7-78fdb7ab120f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.481116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eb1c7da0-c652-4c31-80c7-78fdb7ab120f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044908 Aug 24 10:24:07.481780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 510/883] 10.222.0.86 () {60 vars in 1343 bytes} [Wed Aug 24 10:24:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:24:07.486382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.486861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.492422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.492906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.493341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.494021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.494474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.494895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.515992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:07.523810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4712964e-5b11-49c9-a110-3382fb7bf6e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1562 microversion: 2.1 time: 0.156405 Aug 24 10:24:07.524723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 374/884] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:07 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1562 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:07.529399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.530175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.535955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.536493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.536928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.537619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.538097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.538522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.575737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'services' on Instance uuid bdb467be-865b-489c-ab6c-f714a02672e1 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:07.618490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:07.648378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0fc7a37-89f2-4575-b047-147dd667b4ce) Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac HTTP/1.1 Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:07.650792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:24:07.652397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0fc7a37-89f2-4575-b047-147dd667b4ce) Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': 'bdb467be-865b-489c-ab6c-f714a02672e1', 'id': '57062e38-4dec-4132-969b-b4e85fc3b9ac'} Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.653393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0fc7a37-89f2-4575-b047-147dd667b4ce) Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0fc7a37-89f2-4575-b047-147dd667b4ce) Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.658884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0fc7a37-89f2-4575-b047-147dd667b4ce) Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.660617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0fc7a37-89f2-4575-b047-147dd667b4ce) Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0fc7a37-89f2-4575-b047-147dd667b4ce) Aug 24 10:24:07.662139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.663520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0fc7a37-89f2-4575-b047-147dd667b4ce) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:07.663520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8bbb96db-9311-42e7-bd97-5d78a54a1ec3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac" status: 400 len: 269 microversion: 2.1 time: 0.169507 Aug 24 10:24:07.663520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 511/885] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments/57062e38-4dec-4132-969b-b4e85fc3b9ac => generated 269 bytes in 171 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:07.663520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.663977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.666183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.666688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.667114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.667815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.668303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.668737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.729675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ce22c1e3-5a0e-4096-87ee-a0e44f0e526b) Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:07.731986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:07.733335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ce22c1e3-5a0e-4096-87ee-a0e44f0e526b) Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.734222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ce22c1e3-5a0e-4096-87ee-a0e44f0e526b) Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.735629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ce22c1e3-5a0e-4096-87ee-a0e44f0e526b) Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.737183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ce22c1e3-5a0e-4096-87ee-a0e44f0e526b) Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.738209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ce22c1e3-5a0e-4096-87ee-a0e44f0e526b) Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ce22c1e3-5a0e-4096-87ee-a0e44f0e526b) Aug 24 10:24:07.739480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.741049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ce22c1e3-5a0e-4096-87ee-a0e44f0e526b) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:07.741049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-22394269-298d-4f3e-9513-3e2d2a88ca9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.207765 Aug 24 10:24:07.741049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 375/886] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 269 bytes in 209 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:07.741049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.741417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.745260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.745746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.746181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.746904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.747357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.747782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.760960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:07.762565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.764179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:07.765581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:07.767032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.768539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:07.768539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-30fb60a8-1d9f-4545-959c-163dad27e7bc tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.028924 Aug 24 10:24:07.768539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 376/887] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:24:07.769568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.770069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.775980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.776578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.777151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.777924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.778452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.779087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.784735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:07.806708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:24:07.841725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3fb276bf-00bd-427d-8d22-0521dca014b8) Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:07.844615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:07.846420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3fb276bf-00bd-427d-8d22-0521dca014b8) Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.847133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3fb276bf-00bd-427d-8d22-0521dca014b8) Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:07.848422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3fb276bf-00bd-427d-8d22-0521dca014b8) Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.849832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3fb276bf-00bd-427d-8d22-0521dca014b8) Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.851054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3fb276bf-00bd-427d-8d22-0521dca014b8) Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3fb276bf-00bd-427d-8d22-0521dca014b8) Aug 24 10:24:07.852587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:07.853789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3fb276bf-00bd-427d-8d22-0521dca014b8) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:07.853789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-90c8f107-b1a6-4ac0-b4ff-e5f277c29239 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.192314 Aug 24 10:24:07.853789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 512/888] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 269 bytes in 193 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:07.884932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-acc25ca4-fa47-41c4-bab1-a6cdb60ad536 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 204 len: 0 microversion: 2.1 time: 0.117840 Aug 24 10:24:07.885370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 377/889] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:24:07.891446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cbc86946-3585-4a43-a52d-b9e1aeff835d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.894200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cbc86946-3585-4a43-a52d-b9e1aeff835d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.900522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbc86946-3585-4a43-a52d-b9e1aeff835d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.901279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbc86946-3585-4a43-a52d-b9e1aeff835d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.901711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbc86946-3585-4a43-a52d-b9e1aeff835d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.902435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbc86946-3585-4a43-a52d-b9e1aeff835d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.902887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbc86946-3585-4a43-a52d-b9e1aeff835d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.903306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbc86946-3585-4a43-a52d-b9e1aeff835d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.912319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-a8a0cfd1-23e9-4c73-ab60-d325d4aff52d req-e1ef11d6-7d3b-4c2c-8b38-0ae5f84e8226 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0c601f9d-f26b-4095-a00c-53c422163d31", "tag": "daf8264d-e786-4857-bf7e-3e865d7b48a2"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:24:07.923660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a8a0cfd1-23e9-4c73-ab60-d325d4aff52d req-e1ef11d6-7d3b-4c2c-8b38-0ae5f84e8226 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.924176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a8a0cfd1-23e9-4c73-ab60-d325d4aff52d req-e1ef11d6-7d3b-4c2c-8b38-0ae5f84e8226 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.924609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a8a0cfd1-23e9-4c73-ab60-d325d4aff52d req-e1ef11d6-7d3b-4c2c-8b38-0ae5f84e8226 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.934679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cbc86946-3585-4a43-a52d-b9e1aeff835d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048005 Aug 24 10:24:07.935075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-a8a0cfd1-23e9-4c73-ab60-d325d4aff52d req-e1ef11d6-7d3b-4c2c-8b38-0ae5f84e8226 service nova] Creating event network-changed:daf8264d-e786-4857-bf7e-3e865d7b48a2 for instance 0c601f9d-f26b-4095-a00c-53c422163d31 on hv-cinder-85427 Aug 24 10:24:07.935841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 378/890] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:07 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:07.940890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-a8a0cfd1-23e9-4c73-ab60-d325d4aff52d req-e1ef11d6-7d3b-4c2c-8b38-0ae5f84e8226 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092867 Aug 24 10:24:07.941037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.941469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 513/891] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:24:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:07.941602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.946536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:07.947126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:07.950171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.951296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.951727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.953059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.953518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.953631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.953932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.954115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.954554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:07.955242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:07.955700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:07.956176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.036810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:08.075314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:08.103364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1d9f24a3-dc7a-4420-af4a-5bb38c830b28) Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:08.105813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:08.107432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1d9f24a3-dc7a-4420-af4a-5bb38c830b28) Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.108476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1d9f24a3-dc7a-4420-af4a-5bb38c830b28) Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1d9f24a3-dc7a-4420-af4a-5bb38c830b28) Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.111151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1d9f24a3-dc7a-4420-af4a-5bb38c830b28) Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.112274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1d9f24a3-dc7a-4420-af4a-5bb38c830b28) Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1d9f24a3-dc7a-4420-af4a-5bb38c830b28) Aug 24 10:24:08.113452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.114918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1d9f24a3-dc7a-4420-af4a-5bb38c830b28) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.114918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8e061057-e61f-4c37-8723-4d5f7d4fbb0a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.171957 Aug 24 10:24:08.114918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 379/892] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 269 bytes in 173 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:08.114918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.115301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.121281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.122001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.122429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.123161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.123617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.124702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.138802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:08.141066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.142645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.143752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.145282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.146779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.146779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-783c5c13-3462-45b5-899e-b0d0f11d868c tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.031824 Aug 24 10:24:08.146779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 380/893] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:24:08.147167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-db6c3bd8-8588-4446-ae90-8ad99cde8f43) Aug 24 10:24:08.149102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.149604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:08.149715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:08.151046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-db6c3bd8-8588-4446-ae90-8ad99cde8f43) Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.151710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-db6c3bd8-8588-4446-ae90-8ad99cde8f43) Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.152817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-db6c3bd8-8588-4446-ae90-8ad99cde8f43) Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.154401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-db6c3bd8-8588-4446-ae90-8ad99cde8f43) Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.155736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-db6c3bd8-8588-4446-ae90-8ad99cde8f43) Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-db6c3bd8-8588-4446-ae90-8ad99cde8f43) Aug 24 10:24:08.157101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.158363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-db6c3bd8-8588-4446-ae90-8ad99cde8f43) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.158363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-94b61a8b-906e-4842-b128-7a1c4d810dfc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.208533 Aug 24 10:24:08.158363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 514/894] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:07 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 269 bytes in 210 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:08.158634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.158634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.158634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.158634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.158974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.158974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.161636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-81d74fc2-1af7-4d36-8376-d085881b5371 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.163833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-81d74fc2-1af7-4d36-8376-d085881b5371 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.170680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81d74fc2-1af7-4d36-8376-d085881b5371 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.171185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81d74fc2-1af7-4d36-8376-d085881b5371 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.171661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81d74fc2-1af7-4d36-8376-d085881b5371 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.172398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81d74fc2-1af7-4d36-8376-d085881b5371 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.172863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81d74fc2-1af7-4d36-8376-d085881b5371 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.173295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81d74fc2-1af7-4d36-8376-d085881b5371 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.184983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:24:08.208455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-81d74fc2-1af7-4d36-8376-d085881b5371 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055369 Aug 24 10:24:08.209486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 515/895] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:08.214502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.215010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.222675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.223188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.223638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.224383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.224843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.225277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.232083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:08.237595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.compute.api [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Instance is already in deleting state, ignoring this request Aug 24 10:24:08.238655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-66accbdf-54dc-4fc5-b5a0-50bd8a9f9a16 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 204 len: 0 microversion: 2.1 time: 0.094781 Aug 24 10:24:08.239233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 381/896] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:24:08.244794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.245547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.253191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.253907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.254348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.255063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.255520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.255949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.315448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:08.362045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:08.391921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e90088a6-291f-4525-9254-d988b51d6c0f) Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:08.394203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:08.395711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e90088a6-291f-4525-9254-d988b51d6c0f) Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.396873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e90088a6-291f-4525-9254-d988b51d6c0f) Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.398914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e90088a6-291f-4525-9254-d988b51d6c0f) Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.401198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e90088a6-291f-4525-9254-d988b51d6c0f) Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.403206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e90088a6-291f-4525-9254-d988b51d6c0f) Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e90088a6-291f-4525-9254-d988b51d6c0f) Aug 24 10:24:08.406970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.409517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e90088a6-291f-4525-9254-d988b51d6c0f) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.409517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-73e8354b-424d-44ad-b078-9ff452639ec2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.185324 Aug 24 10:24:08.409517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 516/897] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 269 bytes in 187 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:08.409517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.410840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.414241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.415154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.416317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.417050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.417381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.417872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.432292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e37bdcae-e7c0-4377-8256-8f0123f53860) Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:08.434645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:08.436239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e37bdcae-e7c0-4377-8256-8f0123f53860) Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.438048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e37bdcae-e7c0-4377-8256-8f0123f53860) Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.439195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e37bdcae-e7c0-4377-8256-8f0123f53860) Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.440502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e37bdcae-e7c0-4377-8256-8f0123f53860) Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.442403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e37bdcae-e7c0-4377-8256-8f0123f53860) Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e37bdcae-e7c0-4377-8256-8f0123f53860) Aug 24 10:24:08.444606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.448117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e37bdcae-e7c0-4377-8256-8f0123f53860) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.448117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9cfb02c9-88d9-40c8-b3b6-d015e75741fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.195827 Aug 24 10:24:08.448117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 382/898] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 269 bytes in 197 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:08.448551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.451463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.453046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.455726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.459397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.459541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.459790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fb45fbcc-f63d-4cbd-b303-c6769594f8ef tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.047143 Aug 24 10:24:08.459942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.460101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 517/899] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:24:08.460101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-08a7e4dd-5944-4084-bf6a-e85491ae04da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.460101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-08a7e4dd-5944-4084-bf6a-e85491ae04da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.460514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.460514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.460514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.460514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.461040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.461040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.461958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a7e4dd-5944-4084-bf6a-e85491ae04da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.461958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a7e4dd-5944-4084-bf6a-e85491ae04da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.461958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a7e4dd-5944-4084-bf6a-e85491ae04da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.462520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a7e4dd-5944-4084-bf6a-e85491ae04da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.463125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a7e4dd-5944-4084-bf6a-e85491ae04da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.463586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-08a7e4dd-5944-4084-bf6a-e85491ae04da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.493308 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:24:08.495889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-08a7e4dd-5944-4084-bf6a-e85491ae04da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048740 Aug 24 10:24:08.497031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 518/900] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:08.502366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.502936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.508769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.509366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.509884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.510592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.511124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.511556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.545283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.compute.api [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Instance is already in deleting state, ignoring this request Aug 24 10:24:08.546687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-60059321-00d6-4d97-b7dd-0292ae809d01 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 204 len: 0 microversion: 2.1 time: 0.107462 Aug 24 10:24:08.547217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 383/901] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:24:08.554409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.554901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.560824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.561311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.562383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.563228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.563682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.564152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.597405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:08.652098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lazy-loading 'services' on Instance uuid 987654b5-b3b6-434c-a284-d630ab8ab4ad {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:08.686868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fa4400f-aedb-41eb-8376-3cc7dffc278d) Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:08.688928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:08.690426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fa4400f-aedb-41eb-8376-3cc7dffc278d) Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.691313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fa4400f-aedb-41eb-8376-3cc7dffc278d) Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.692416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fa4400f-aedb-41eb-8376-3cc7dffc278d) Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.693643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fa4400f-aedb-41eb-8376-3cc7dffc278d) Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.696235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fa4400f-aedb-41eb-8376-3cc7dffc278d) Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fa4400f-aedb-41eb-8376-3cc7dffc278d) Aug 24 10:24:08.698452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.701376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fa4400f-aedb-41eb-8376-3cc7dffc278d) {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.701376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-07b23c6f-77ee-4eae-a0c9-211ec6db83bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.192456 Aug 24 10:24:08.701376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 519/902] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 269 bytes in 194 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:08.701376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.701918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.706316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.706316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.706316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.707591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.708276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.708975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.725520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-39b55200-06a1-4e32-b97f-a65ec6d1f4ae) Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Request method failure captured: Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e HTTP/1.1 Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBffmjbTNpJugu_xz5VX3-6C7n22Er8fBdh46ha1xwUrJ9NCiGPhjpWbhGC8XIyL8p2EH-mr3pW-jcN5-qKck-aTemDWyejGV9S8Ozx6ZF3Zl6sOWZLe-FxBlVrszp2XDg-2UUXu3GxlbJCJjzR1KMBm2mZp8RGhkqhTH7I6jEhzwaiE Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:08.727815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/5294ff8dc07444e5b8758dbae2db7fc4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 5294ff8dc07444e5b8758dbae2db7fc4 Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-TestMinimumBasicScenario-460256604 Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: bea2aecac254429a823b8765feb6c767 Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-TestMinimumBasicScenario-460256604-project Aug 24 10:24:08.729266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-39b55200-06a1-4e32-b97f-a65ec6d1f4ae) Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '987654b5-b3b6-434c-a284-d630ab8ab4ad', 'id': '53ed8765-f72f-48b6-8b3c-c9f975174c7e'} Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.729928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-39b55200-06a1-4e32-b97f-a65ec6d1f4ae) Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:08.730985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-39b55200-06a1-4e32-b97f-a65ec6d1f4ae) Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.732203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-39b55200-06a1-4e32-b97f-a65ec6d1f4ae) Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.733242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-39b55200-06a1-4e32-b97f-a65ec6d1f4ae) Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-39b55200-06a1-4e32-b97f-a65ec6d1f4ae) Aug 24 10:24:08.734927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.735989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-39b55200-06a1-4e32-b97f-a65ec6d1f4ae) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.737402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.737565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bb12834f-491d-4da7-9ab4-086834fa7d9d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e" status: 400 len: 269 microversion: 2.1 time: 0.181005 Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:08.737704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.741768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.743209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.748135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.763495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 384/903] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad/os-volume_attachments/53ed8765-f72f-48b6-8b3c-c9f975174c7e => generated 269 bytes in 184 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:08.763650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.763650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-87b85a8b-3976-4313-9467-bb750232215b tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.041249 Aug 24 10:24:08.763650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 520/904] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:24:08.764055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.764055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.764055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.764055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.764605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.764751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-df7c1947-5a72-4fef-bfb6-809a9628ab31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.764917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.764917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.764917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.765313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-df7c1947-5a72-4fef-bfb6-809a9628ab31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.765313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df7c1947-5a72-4fef-bfb6-809a9628ab31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.765313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df7c1947-5a72-4fef-bfb6-809a9628ab31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.765313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df7c1947-5a72-4fef-bfb6-809a9628ab31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.765839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df7c1947-5a72-4fef-bfb6-809a9628ab31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.765839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df7c1947-5a72-4fef-bfb6-809a9628ab31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.765839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df7c1947-5a72-4fef-bfb6-809a9628ab31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.789783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:24:08.815045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:08.822770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-df7c1947-5a72-4fef-bfb6-809a9628ab31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080763 Aug 24 10:24:08.823557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 521/905] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:08.840506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8f587967-73c0-497d-8056-183e6cc7bf18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.840506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8f587967-73c0-497d-8056-183e6cc7bf18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.845756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.compute.api [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Instance is already in deleting state, ignoring this request Aug 24 10:24:08.846633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-17f89d09-bc14-4eb0-8632-b1fd4ed141b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 204 len: 0 microversion: 2.1 time: 0.113394 Aug 24 10:24:08.847326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 385/906] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:24:08.853517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f587967-73c0-497d-8056-183e6cc7bf18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.853924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.854424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.856119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f587967-73c0-497d-8056-183e6cc7bf18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.856667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f587967-73c0-497d-8056-183e6cc7bf18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.857401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f587967-73c0-497d-8056-183e6cc7bf18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.857864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f587967-73c0-497d-8056-183e6cc7bf18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.859753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f587967-73c0-497d-8056-183e6cc7bf18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.862406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.863074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.863698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.865964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.866675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.867316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.897937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8f587967-73c0-497d-8056-183e6cc7bf18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073380 Aug 24 10:24:08.898821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 522/907] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:08 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:08.907693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.908778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] [instance: 350e1445-9336-439c-b146-ee4c74211b54] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.916852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.917365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.918064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.918871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.919402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.919959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.934793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] HTTP exception thrown: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Request method failure captured: Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 HTTP/1.1 Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBffQZO1Tj9lsmcY4e3x-MTKwEouDGMLH8fop0WjuEUzltswWzSID9sbaKMKpPtu2OI2OvohMmR3xOzCQ0RZ3WC2wJVtbvltk3eX6GWaA1FvYKTXl0nft8eZDK9AEvbBaKCljhyYYpGV8-3N1zh7cMhUiY_mLh5EveM0CFCq71dy67JY Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:24:08.936599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/dc33f347b062491aaa04b7dc18553cf4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: dc33f347b062491aaa04b7dc18553cf4 Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-TestVolumeBootPattern-1762380404 Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 3ccfd2f7e35646b0a60b28b6807cee4b Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-TestVolumeBootPattern-1762380404-project Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.938169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '350e1445-9336-439c-b146-ee4c74211b54'} Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:08.938916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. Aug 24 10:24:08.940175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:24:08.942483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Returning 404 to user: Instance 350e1445-9336-439c-b146-ee4c74211b54 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:08.943493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-92297c1e-81d6-4845-8a73-48b8a60a4016 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54" status: 404 len: 111 microversion: 2.1 time: 0.043419 Aug 24 10:24:08.944057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 523/908] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:08 2022] GET /compute/v2.1/servers/350e1445-9336-439c-b146-ee4c74211b54 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:24:08.948825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:08.949748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:08.955854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.956425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.956868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.957792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:08.958854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:08.959288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:08.985674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:24:09.112800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:09.124788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6d8b7d79-1fe9-4e58-9ee3-df651e2cd1ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1500 microversion: 2.1 time: 0.179390 Aug 24 10:24:09.125482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 524/909] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:08 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1500 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:09.163182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-30f6c93d-5751-4ac7-bab0-fe9fdaa8bae4) Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f HTTP/1.1 Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:24:09.165232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:24:09.166754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-30f6c93d-5751-4ac7-bab0-fe9fdaa8bae4) Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '0c3cf0b5-40f3-40f9-83ec-6de8d63aca02', 'id': 'c33defcd-3e83-4c15-bd33-d4758c15058f'} Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:09.167610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-30f6c93d-5751-4ac7-bab0-fe9fdaa8bae4) Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5177, in _detach_volume Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 24 10:24:09.169058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-30f6c93d-5751-4ac7-bab0-fe9fdaa8bae4) Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:09.170483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5199, in detach_volume Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5179, in _detach_volume Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-30f6c93d-5751-4ac7-bab0-fe9fdaa8bae4) Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:09.172908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-30f6c93d-5751-4ac7-bab0-fe9fdaa8bae4) Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-30f6c93d-5751-4ac7-bab0-fe9fdaa8bae4) Aug 24 10:24:09.174748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:24:09.177705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-30f6c93d-5751-4ac7-bab0-fe9fdaa8bae4) {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:24:09.177705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-51c8bb78-8baa-431f-af8c-9a3ab0c72de2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f" status: 400 len: 269 microversion: 2.1 time: 0.318473 Aug 24 10:24:09.177705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 386/910] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:24:08 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02/os-volume_attachments/c33defcd-3e83-4c15-bd33-d4758c15058f => generated 269 bytes in 320 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:24:09.216485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-ad1e2a95-9701-419e-aa82-9dcf88930f6e req-3563c4d0-d542-4108-a822-69ff2c9fcacc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0c601f9d-f26b-4095-a00c-53c422163d31", "name": "network-vif-deleted", "tag": "daf8264d-e786-4857-bf7e-3e865d7b48a2"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:24:09.226192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-ad1e2a95-9701-419e-aa82-9dcf88930f6e req-3563c4d0-d542-4108-a822-69ff2c9fcacc service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:09.226983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-ad1e2a95-9701-419e-aa82-9dcf88930f6e req-3563c4d0-d542-4108-a822-69ff2c9fcacc service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:09.227422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-ad1e2a95-9701-419e-aa82-9dcf88930f6e req-3563c4d0-d542-4108-a822-69ff2c9fcacc service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:09.240616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-ad1e2a95-9701-419e-aa82-9dcf88930f6e req-3563c4d0-d542-4108-a822-69ff2c9fcacc service nova] Creating event network-vif-deleted:daf8264d-e786-4857-bf7e-3e865d7b48a2 for instance 0c601f9d-f26b-4095-a00c-53c422163d31 on hv-cinder-85427 Aug 24 10:24:09.245804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-ad1e2a95-9701-419e-aa82-9dcf88930f6e req-3563c4d0-d542-4108-a822-69ff2c9fcacc service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.119014 Aug 24 10:24:09.246337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 525/911] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:24:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:09.837024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dd02db38-d183-4ca0-ade8-5f55002574ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:09.838023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dd02db38-d183-4ca0-ade8-5f55002574ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:09.847697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd02db38-d183-4ca0-ade8-5f55002574ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:09.848217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd02db38-d183-4ca0-ade8-5f55002574ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:09.848644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd02db38-d183-4ca0-ade8-5f55002574ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:09.849346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd02db38-d183-4ca0-ade8-5f55002574ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:09.849796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd02db38-d183-4ca0-ade8-5f55002574ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:09.850216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd02db38-d183-4ca0-ade8-5f55002574ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:09.879537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dd02db38-d183-4ca0-ade8-5f55002574ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048414 Aug 24 10:24:09.880109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 387/912] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:09 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:09.911584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-977c4aed-a6f2-4d72-824b-b3e6f192e8ff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:09.913970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-977c4aed-a6f2-4d72-824b-b3e6f192e8ff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:09.920263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-977c4aed-a6f2-4d72-824b-b3e6f192e8ff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:09.920759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-977c4aed-a6f2-4d72-824b-b3e6f192e8ff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:09.921194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-977c4aed-a6f2-4d72-824b-b3e6f192e8ff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:09.921901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-977c4aed-a6f2-4d72-824b-b3e6f192e8ff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:09.922360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-977c4aed-a6f2-4d72-824b-b3e6f192e8ff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:09.922785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-977c4aed-a6f2-4d72-824b-b3e6f192e8ff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:09.953630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-977c4aed-a6f2-4d72-824b-b3e6f192e8ff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047406 Aug 24 10:24:09.954189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 526/913] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:09 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:10.150299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:10.150944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:10.157966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:10.158438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:10.158875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:10.159573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:10.160030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:10.160484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:10.226692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:10.234579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c110f0b8-19b2-4950-b172-796b7996d42f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.088120 Aug 24 10:24:10.235144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 388/914] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:10 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:10.891847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-034f0292-1edb-4bfc-959d-ba977544eabf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:10.894013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-034f0292-1edb-4bfc-959d-ba977544eabf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:10.899912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034f0292-1edb-4bfc-959d-ba977544eabf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:10.900434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034f0292-1edb-4bfc-959d-ba977544eabf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:10.900877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034f0292-1edb-4bfc-959d-ba977544eabf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:10.901573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034f0292-1edb-4bfc-959d-ba977544eabf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:10.902042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034f0292-1edb-4bfc-959d-ba977544eabf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:10.902477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034f0292-1edb-4bfc-959d-ba977544eabf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:10.932400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-034f0292-1edb-4bfc-959d-ba977544eabf tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044545 Aug 24 10:24:10.932995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 527/915] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:10 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:10.966008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-74adafd6-0951-47fb-8cc1-1693ec287e4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:10.968771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-74adafd6-0951-47fb-8cc1-1693ec287e4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:10.975167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74adafd6-0951-47fb-8cc1-1693ec287e4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:10.975655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74adafd6-0951-47fb-8cc1-1693ec287e4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:10.976116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74adafd6-0951-47fb-8cc1-1693ec287e4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:10.976805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74adafd6-0951-47fb-8cc1-1693ec287e4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:10.977260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74adafd6-0951-47fb-8cc1-1693ec287e4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:10.977682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74adafd6-0951-47fb-8cc1-1693ec287e4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:11.006889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-74adafd6-0951-47fb-8cc1-1693ec287e4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044847 Aug 24 10:24:11.007428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 389/916] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:10 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:11.254744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:11.255363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:11.261931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:11.262427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:11.262868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:11.263618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:11.264093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:11.264525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:11.326430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:11.334919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5aa77be9-5f82-46e0-9bf6-4af253f86947 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084032 Aug 24 10:24:11.335467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 528/917] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:11 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:11.945146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d71b998c-aa13-4624-ba94-f92827a6556f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:11.947291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d71b998c-aa13-4624-ba94-f92827a6556f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:11.953248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d71b998c-aa13-4624-ba94-f92827a6556f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:11.954313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d71b998c-aa13-4624-ba94-f92827a6556f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:11.954755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d71b998c-aa13-4624-ba94-f92827a6556f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:11.955448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d71b998c-aa13-4624-ba94-f92827a6556f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:11.955908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d71b998c-aa13-4624-ba94-f92827a6556f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:11.956371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d71b998c-aa13-4624-ba94-f92827a6556f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:11.985668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d71b998c-aa13-4624-ba94-f92827a6556f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044521 Aug 24 10:24:11.986286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 390/918] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:11 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:12.019428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e4f00b8d-06b8-46dc-b6cf-4d8a57ed307c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:12.021645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e4f00b8d-06b8-46dc-b6cf-4d8a57ed307c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:12.027868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4f00b8d-06b8-46dc-b6cf-4d8a57ed307c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:12.028386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4f00b8d-06b8-46dc-b6cf-4d8a57ed307c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:12.028822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4f00b8d-06b8-46dc-b6cf-4d8a57ed307c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:12.029506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4f00b8d-06b8-46dc-b6cf-4d8a57ed307c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:12.029956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4f00b8d-06b8-46dc-b6cf-4d8a57ed307c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:12.030379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4f00b8d-06b8-46dc-b6cf-4d8a57ed307c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:12.058949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e4f00b8d-06b8-46dc-b6cf-4d8a57ed307c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044091 Aug 24 10:24:12.059490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 529/919] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:12 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:12.355555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:12.356108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:12.362635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:12.363111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:12.363542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:12.364261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:12.364715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:12.365137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:12.426558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:12.434164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-464cbf79-089f-4017-8515-3e7cb8f53b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082628 Aug 24 10:24:12.434733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 391/920] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:12 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:12.997588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5c77e390-2361-4fd9-9858-2a192104cee1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:12.999734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5c77e390-2361-4fd9-9858-2a192104cee1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:13.005736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c77e390-2361-4fd9-9858-2a192104cee1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:13.006224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c77e390-2361-4fd9-9858-2a192104cee1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:13.006669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c77e390-2361-4fd9-9858-2a192104cee1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:13.007356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c77e390-2361-4fd9-9858-2a192104cee1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:13.007812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c77e390-2361-4fd9-9858-2a192104cee1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:13.008283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c77e390-2361-4fd9-9858-2a192104cee1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:13.039620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5c77e390-2361-4fd9-9858-2a192104cee1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045959 Aug 24 10:24:13.040297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 530/921] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:13.071745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aa2c7050-551b-4b51-90ff-388ba420c5bc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:13.073950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-aa2c7050-551b-4b51-90ff-388ba420c5bc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:13.080882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa2c7050-551b-4b51-90ff-388ba420c5bc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:13.081365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa2c7050-551b-4b51-90ff-388ba420c5bc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:13.081803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa2c7050-551b-4b51-90ff-388ba420c5bc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:13.082486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa2c7050-551b-4b51-90ff-388ba420c5bc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:13.082942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa2c7050-551b-4b51-90ff-388ba420c5bc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:13.083635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa2c7050-551b-4b51-90ff-388ba420c5bc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:13.112887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aa2c7050-551b-4b51-90ff-388ba420c5bc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045206 Aug 24 10:24:13.113426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 392/922] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:13 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:13.456671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:13.457282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:13.463386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:13.463861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:13.464330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:13.465008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:13.465456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:13.465884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:13.525287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:13.533040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d329d090-d0e1-4839-af6e-0ebbf614a4c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080940 Aug 24 10:24:13.533595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 531/923] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:13 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:14.051572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-98fded97-d213-4724-8a47-cf8aa9845dd2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:14.053957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-98fded97-d213-4724-8a47-cf8aa9845dd2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:14.059803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98fded97-d213-4724-8a47-cf8aa9845dd2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:14.060310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98fded97-d213-4724-8a47-cf8aa9845dd2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:14.060739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98fded97-d213-4724-8a47-cf8aa9845dd2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:14.061451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98fded97-d213-4724-8a47-cf8aa9845dd2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:14.061897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98fded97-d213-4724-8a47-cf8aa9845dd2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:14.062680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98fded97-d213-4724-8a47-cf8aa9845dd2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:14.091769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-98fded97-d213-4724-8a47-cf8aa9845dd2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044834 Aug 24 10:24:14.092450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 393/924] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:14 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:14.125676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d05765c4-5348-4bb5-b313-c15be23edd04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:14.127829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d05765c4-5348-4bb5-b313-c15be23edd04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:14.134270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d05765c4-5348-4bb5-b313-c15be23edd04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:14.134792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d05765c4-5348-4bb5-b313-c15be23edd04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:14.135236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d05765c4-5348-4bb5-b313-c15be23edd04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:14.135937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d05765c4-5348-4bb5-b313-c15be23edd04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:14.136427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d05765c4-5348-4bb5-b313-c15be23edd04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:14.136861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d05765c4-5348-4bb5-b313-c15be23edd04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:14.165489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d05765c4-5348-4bb5-b313-c15be23edd04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043785 Aug 24 10:24:14.166032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 532/925] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:14 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:14.558664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:14.559280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:14.565236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:14.565726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:14.566188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:14.566880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:14.567335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:14.567765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:14.629706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:14.633150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:14.638226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-13bdba6d-634f-4f29-829f-ca67b5f37e5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083519 Aug 24 10:24:14.638786 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 394/926] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:14 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:15.104087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3b7ea02b-0ee3-4675-9636-7f4a22d9023d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:15.106151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3b7ea02b-0ee3-4675-9636-7f4a22d9023d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:15.112740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b7ea02b-0ee3-4675-9636-7f4a22d9023d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:15.113227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b7ea02b-0ee3-4675-9636-7f4a22d9023d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:15.113668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b7ea02b-0ee3-4675-9636-7f4a22d9023d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:15.114362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b7ea02b-0ee3-4675-9636-7f4a22d9023d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:15.114822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b7ea02b-0ee3-4675-9636-7f4a22d9023d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:15.115255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b7ea02b-0ee3-4675-9636-7f4a22d9023d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:15.145311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3b7ea02b-0ee3-4675-9636-7f4a22d9023d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045192 Aug 24 10:24:15.146099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 533/927] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:15 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:15.179038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-df1ebc3a-15e8-4302-bc12-3c139db55113 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:15.181228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-df1ebc3a-15e8-4302-bc12-3c139db55113 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:15.188560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df1ebc3a-15e8-4302-bc12-3c139db55113 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:15.189043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df1ebc3a-15e8-4302-bc12-3c139db55113 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:15.189478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df1ebc3a-15e8-4302-bc12-3c139db55113 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:15.190169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df1ebc3a-15e8-4302-bc12-3c139db55113 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:15.190631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df1ebc3a-15e8-4302-bc12-3c139db55113 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:15.191121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df1ebc3a-15e8-4302-bc12-3c139db55113 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:15.221153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-df1ebc3a-15e8-4302-bc12-3c139db55113 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047008 Aug 24 10:24:15.221704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 395/928] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:15 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:15.659696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:15.660327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:15.666225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:15.666707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:15.667140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:15.667821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:15.668294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:15.668715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:15.732268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:15.740645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d662c680-c4f7-4baf-9654-83d50c0226ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085347 Aug 24 10:24:15.741191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 534/929] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:15 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:16.158616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d22f70a7-5dab-4386-b1d1-bd36271e9afb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:16.160783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d22f70a7-5dab-4386-b1d1-bd36271e9afb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:16.166955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d22f70a7-5dab-4386-b1d1-bd36271e9afb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:16.167435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d22f70a7-5dab-4386-b1d1-bd36271e9afb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:16.167869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d22f70a7-5dab-4386-b1d1-bd36271e9afb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:16.168590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d22f70a7-5dab-4386-b1d1-bd36271e9afb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:16.169042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d22f70a7-5dab-4386-b1d1-bd36271e9afb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:16.169466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d22f70a7-5dab-4386-b1d1-bd36271e9afb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:16.198528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d22f70a7-5dab-4386-b1d1-bd36271e9afb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043861 Aug 24 10:24:16.199225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 396/930] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:16 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:16.233273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0d38be49-5887-47bc-a74b-7f1f853964c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:16.235378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0d38be49-5887-47bc-a74b-7f1f853964c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:16.241871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d38be49-5887-47bc-a74b-7f1f853964c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:16.242347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d38be49-5887-47bc-a74b-7f1f853964c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:16.242782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d38be49-5887-47bc-a74b-7f1f853964c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:16.243463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d38be49-5887-47bc-a74b-7f1f853964c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:16.243913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d38be49-5887-47bc-a74b-7f1f853964c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:16.244361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d38be49-5887-47bc-a74b-7f1f853964c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:16.274396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0d38be49-5887-47bc-a74b-7f1f853964c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044951 Aug 24 10:24:16.274946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 535/931] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:16 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:16.761415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:16.762035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:16.768155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:16.768645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:16.769088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:16.769791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:16.770269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:16.770965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:16.836065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:16.843870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ebd4a241-4495-4dc3-a95a-eb16552afe54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086301 Aug 24 10:24:16.844463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 397/932] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:16 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:17.211352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-19d2e472-3a09-43b9-9db3-3bc233416d4c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:17.213522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-19d2e472-3a09-43b9-9db3-3bc233416d4c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:17.220099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d2e472-3a09-43b9-9db3-3bc233416d4c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:17.220563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d2e472-3a09-43b9-9db3-3bc233416d4c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:17.220995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d2e472-3a09-43b9-9db3-3bc233416d4c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:17.221682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d2e472-3a09-43b9-9db3-3bc233416d4c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:17.222134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d2e472-3a09-43b9-9db3-3bc233416d4c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:17.222558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d2e472-3a09-43b9-9db3-3bc233416d4c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:17.256952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-19d2e472-3a09-43b9-9db3-3bc233416d4c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049875 Aug 24 10:24:17.257636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 536/933] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:17 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:17.286403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c0502dfc-12d2-443d-be8e-05048c174cff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:17.288636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c0502dfc-12d2-443d-be8e-05048c174cff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:17.292100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:17.296634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0502dfc-12d2-443d-be8e-05048c174cff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:17.297124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0502dfc-12d2-443d-be8e-05048c174cff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:17.297563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0502dfc-12d2-443d-be8e-05048c174cff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:17.298265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0502dfc-12d2-443d-be8e-05048c174cff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:17.298727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0502dfc-12d2-443d-be8e-05048c174cff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:17.299153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0502dfc-12d2-443d-be8e-05048c174cff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:17.330429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c0502dfc-12d2-443d-be8e-05048c174cff tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047977 Aug 24 10:24:17.330978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 398/934] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:17 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:17.865789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:17.866401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:17.872475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:17.873220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:17.873640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:17.874323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:17.875258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:17.875684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:17.936507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:17.944669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-16c8e18c-0e8b-45ba-aa3b-d45ac25e9dff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082809 Aug 24 10:24:17.945216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 537/935] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:17 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:18.269353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-65595262-e66b-45a4-9aa7-5ba6ba82e172 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:18.271534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-65595262-e66b-45a4-9aa7-5ba6ba82e172 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:18.277947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65595262-e66b-45a4-9aa7-5ba6ba82e172 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:18.278487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65595262-e66b-45a4-9aa7-5ba6ba82e172 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:18.278867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65595262-e66b-45a4-9aa7-5ba6ba82e172 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:18.279569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65595262-e66b-45a4-9aa7-5ba6ba82e172 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:18.280075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65595262-e66b-45a4-9aa7-5ba6ba82e172 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:18.280500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65595262-e66b-45a4-9aa7-5ba6ba82e172 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:18.309991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-65595262-e66b-45a4-9aa7-5ba6ba82e172 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044632 Aug 24 10:24:18.310662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 399/936] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:18 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:18.343043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5d62adb2-7cfb-4d4c-874f-f91b73034867 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:18.345405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5d62adb2-7cfb-4d4c-874f-f91b73034867 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:18.348896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:18.352282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d62adb2-7cfb-4d4c-874f-f91b73034867 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:18.352771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d62adb2-7cfb-4d4c-874f-f91b73034867 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:18.353217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d62adb2-7cfb-4d4c-874f-f91b73034867 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:18.353920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d62adb2-7cfb-4d4c-874f-f91b73034867 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:18.354384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d62adb2-7cfb-4d4c-874f-f91b73034867 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:18.354814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d62adb2-7cfb-4d4c-874f-f91b73034867 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:18.384879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5d62adb2-7cfb-4d4c-874f-f91b73034867 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045769 Aug 24 10:24:18.385418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 538/937] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:18 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:18.965072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:18.965677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:18.972255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:18.972730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:18.973159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:18.973833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:18.974277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:18.974690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:19.034771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:19.042374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a4d83c7e-e6f7-40f8-affc-248e3c822bfd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081163 Aug 24 10:24:19.042923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 400/938] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:18 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:19.323419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-201cf219-be25-4dad-bd9f-1031b8c0a2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:19.325577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-201cf219-be25-4dad-bd9f-1031b8c0a2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:19.332243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201cf219-be25-4dad-bd9f-1031b8c0a2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:19.332734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201cf219-be25-4dad-bd9f-1031b8c0a2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:19.333448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201cf219-be25-4dad-bd9f-1031b8c0a2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:19.334144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201cf219-be25-4dad-bd9f-1031b8c0a2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:19.334612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201cf219-be25-4dad-bd9f-1031b8c0a2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:19.335046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201cf219-be25-4dad-bd9f-1031b8c0a2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:19.359736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:19.364940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-201cf219-be25-4dad-bd9f-1031b8c0a2a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045694 Aug 24 10:24:19.365498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 539/939] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:19 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:19.397671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3b5e2fa2-c4a1-4a44-9d02-aa804847421a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:19.399831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3b5e2fa2-c4a1-4a44-9d02-aa804847421a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:19.406244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b5e2fa2-c4a1-4a44-9d02-aa804847421a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:19.406727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b5e2fa2-c4a1-4a44-9d02-aa804847421a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:19.407169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b5e2fa2-c4a1-4a44-9d02-aa804847421a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:19.407847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b5e2fa2-c4a1-4a44-9d02-aa804847421a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:19.408330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b5e2fa2-c4a1-4a44-9d02-aa804847421a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:19.408758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b5e2fa2-c4a1-4a44-9d02-aa804847421a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:19.437094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3b5e2fa2-c4a1-4a44-9d02-aa804847421a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043800 Aug 24 10:24:19.438123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 401/940] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:19 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:20.063276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:20.064028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:20.070250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:20.070751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:20.071199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:20.071966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:20.072478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:20.072933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:20.134397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:20.142014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6dd45789-7372-496d-88c6-27bfd0ae3319 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082995 Aug 24 10:24:20.142569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 540/941] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:20 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:20.377862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-65a3cb1f-d488-4e24-ac30-b849c88dd2b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:20.379906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-65a3cb1f-d488-4e24-ac30-b849c88dd2b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:20.385803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65a3cb1f-d488-4e24-ac30-b849c88dd2b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:20.386280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65a3cb1f-d488-4e24-ac30-b849c88dd2b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:20.386713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65a3cb1f-d488-4e24-ac30-b849c88dd2b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:20.387395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65a3cb1f-d488-4e24-ac30-b849c88dd2b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:20.387846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65a3cb1f-d488-4e24-ac30-b849c88dd2b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:20.388289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65a3cb1f-d488-4e24-ac30-b849c88dd2b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:20.419297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-65a3cb1f-d488-4e24-ac30-b849c88dd2b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045478 Aug 24 10:24:20.419923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 402/942] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:20 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:20.449384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-153a79f6-25fa-4271-ba53-2043b678838d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:20.451507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-153a79f6-25fa-4271-ba53-2043b678838d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:20.458796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-153a79f6-25fa-4271-ba53-2043b678838d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:20.459291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-153a79f6-25fa-4271-ba53-2043b678838d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:20.459736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-153a79f6-25fa-4271-ba53-2043b678838d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:20.460470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-153a79f6-25fa-4271-ba53-2043b678838d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:20.460954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-153a79f6-25fa-4271-ba53-2043b678838d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:20.461385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-153a79f6-25fa-4271-ba53-2043b678838d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:20.490680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-153a79f6-25fa-4271-ba53-2043b678838d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045376 Aug 24 10:24:20.491229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 541/943] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:20 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:21.162891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:21.163501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:21.170066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:21.170594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:21.171035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:21.172279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:21.172737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:21.173194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:21.237627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:21.245548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b5c06be8-7f4f-4038-a774-08ab0cf517da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086513 Aug 24 10:24:21.246107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 403/944] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:21 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:21.431507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e7eeb447-90cc-4ae0-b17a-235a79993b30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:21.433709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e7eeb447-90cc-4ae0-b17a-235a79993b30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:21.439652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7eeb447-90cc-4ae0-b17a-235a79993b30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:21.440173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7eeb447-90cc-4ae0-b17a-235a79993b30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:21.440617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7eeb447-90cc-4ae0-b17a-235a79993b30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:21.441335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7eeb447-90cc-4ae0-b17a-235a79993b30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:21.441798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7eeb447-90cc-4ae0-b17a-235a79993b30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:21.442246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7eeb447-90cc-4ae0-b17a-235a79993b30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:21.471860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e7eeb447-90cc-4ae0-b17a-235a79993b30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044318 Aug 24 10:24:21.472529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 542/945] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:21 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:21.502880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-55446e82-87d9-45a5-8b0f-ba4817ab140f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:21.505145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-55446e82-87d9-45a5-8b0f-ba4817ab140f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:21.511572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55446e82-87d9-45a5-8b0f-ba4817ab140f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:21.512067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55446e82-87d9-45a5-8b0f-ba4817ab140f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:21.512521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55446e82-87d9-45a5-8b0f-ba4817ab140f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:21.513212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55446e82-87d9-45a5-8b0f-ba4817ab140f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:21.513668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55446e82-87d9-45a5-8b0f-ba4817ab140f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:21.514095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55446e82-87d9-45a5-8b0f-ba4817ab140f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:21.543361 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-55446e82-87d9-45a5-8b0f-ba4817ab140f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044459 Aug 24 10:24:21.543909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 404/946] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:21 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:22.267363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:22.267977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:22.274010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:22.274496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:22.274940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:22.275632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:22.276119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:22.276553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:22.336240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:22.343949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a2f89b7b-67e8-4fbd-851d-dc60bf77a600 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081060 Aug 24 10:24:22.344534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 543/947] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:22 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:22.484506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1e6b2876-2c25-4196-90a6-882edd925836 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:22.486729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1e6b2876-2c25-4196-90a6-882edd925836 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:22.492748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6b2876-2c25-4196-90a6-882edd925836 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:22.493232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6b2876-2c25-4196-90a6-882edd925836 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:22.493666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6b2876-2c25-4196-90a6-882edd925836 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:22.494352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6b2876-2c25-4196-90a6-882edd925836 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:22.494806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6b2876-2c25-4196-90a6-882edd925836 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:22.495230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6b2876-2c25-4196-90a6-882edd925836 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:22.525960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1e6b2876-2c25-4196-90a6-882edd925836 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045388 Aug 24 10:24:22.526587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 405/948] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:22 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:22.556019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c2ef5683-52cc-4899-9d55-8329f5e00604 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:22.558162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c2ef5683-52cc-4899-9d55-8329f5e00604 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:22.565046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2ef5683-52cc-4899-9d55-8329f5e00604 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:22.565527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2ef5683-52cc-4899-9d55-8329f5e00604 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:22.565963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2ef5683-52cc-4899-9d55-8329f5e00604 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:22.566648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2ef5683-52cc-4899-9d55-8329f5e00604 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:22.567101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2ef5683-52cc-4899-9d55-8329f5e00604 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:22.567534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2ef5683-52cc-4899-9d55-8329f5e00604 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:22.596278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c2ef5683-52cc-4899-9d55-8329f5e00604 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044199 Aug 24 10:24:22.596815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 544/949] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:22 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:23.364388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:23.365007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:23.370871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:23.371355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:23.371791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:23.372513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:23.372969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:23.373397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:23.434597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:23.442411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-114becc1-aff3-4e8c-af18-81d0671b1a78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081949 Aug 24 10:24:23.442968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 406/950] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:23 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:23.541310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c7f68fc0-f112-4fa8-a3b2-20328ab5a354 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:23.543594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c7f68fc0-f112-4fa8-a3b2-20328ab5a354 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:23.549475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7f68fc0-f112-4fa8-a3b2-20328ab5a354 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:23.549959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7f68fc0-f112-4fa8-a3b2-20328ab5a354 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:23.550394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7f68fc0-f112-4fa8-a3b2-20328ab5a354 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:23.551077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7f68fc0-f112-4fa8-a3b2-20328ab5a354 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:23.551525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7f68fc0-f112-4fa8-a3b2-20328ab5a354 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:23.551946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7f68fc0-f112-4fa8-a3b2-20328ab5a354 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:23.580764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c7f68fc0-f112-4fa8-a3b2-20328ab5a354 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043494 Aug 24 10:24:23.581415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 545/951] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:23 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:23.607740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7110cfe6-542d-4b8f-870f-2e1d7a0b98d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:23.609947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7110cfe6-542d-4b8f-870f-2e1d7a0b98d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:23.616456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7110cfe6-542d-4b8f-870f-2e1d7a0b98d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:23.616940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7110cfe6-542d-4b8f-870f-2e1d7a0b98d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:23.617374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7110cfe6-542d-4b8f-870f-2e1d7a0b98d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:23.618064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7110cfe6-542d-4b8f-870f-2e1d7a0b98d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:23.618521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7110cfe6-542d-4b8f-870f-2e1d7a0b98d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:23.618958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7110cfe6-542d-4b8f-870f-2e1d7a0b98d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:23.648323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7110cfe6-542d-4b8f-870f-2e1d7a0b98d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044585 Aug 24 10:24:23.648861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 407/952] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:23 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:24.463310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:24.463947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:24.470425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:24.470917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:24.471352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:24.472067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:24.472531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:24.472964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:24.536274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:24.543956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a7300db0-6006-4597-aafe-cae835a8ee9b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084549 Aug 24 10:24:24.544533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 546/953] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:24 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:24.594385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-095c1fae-8496-434e-bf7a-e5f93313170e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:24.596591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-095c1fae-8496-434e-bf7a-e5f93313170e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:24.602718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-095c1fae-8496-434e-bf7a-e5f93313170e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:24.603308 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-095c1fae-8496-434e-bf7a-e5f93313170e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:24.603638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-095c1fae-8496-434e-bf7a-e5f93313170e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:24.604379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-095c1fae-8496-434e-bf7a-e5f93313170e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:24.604835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-095c1fae-8496-434e-bf7a-e5f93313170e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:24.605304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-095c1fae-8496-434e-bf7a-e5f93313170e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:24.634798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-095c1fae-8496-434e-bf7a-e5f93313170e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044953 Aug 24 10:24:24.635424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 408/954] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:24 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:24.660883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-62af11e6-eb91-49cf-8903-88e9c73afc72 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:24.663046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-62af11e6-eb91-49cf-8903-88e9c73afc72 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:24.669410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62af11e6-eb91-49cf-8903-88e9c73afc72 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:24.669901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62af11e6-eb91-49cf-8903-88e9c73afc72 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:24.670346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62af11e6-eb91-49cf-8903-88e9c73afc72 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:24.671075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62af11e6-eb91-49cf-8903-88e9c73afc72 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:24.672150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62af11e6-eb91-49cf-8903-88e9c73afc72 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:24.672801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62af11e6-eb91-49cf-8903-88e9c73afc72 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:24.701105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-62af11e6-eb91-49cf-8903-88e9c73afc72 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044210 Aug 24 10:24:24.701655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 547/955] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:24 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:25.565318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:25.565929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:25.571884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:25.572397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:25.572837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:25.573519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:25.573969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:25.574392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:25.637415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:25.645149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e624c593-ea72-46ab-94e6-768ea136bf97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084294 Aug 24 10:24:25.645682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 409/956] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:25 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:25.647749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f053de9f-5bc6-4aae-9c51-b8a972bd5978 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:25.650140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f053de9f-5bc6-4aae-9c51-b8a972bd5978 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:25.656285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f053de9f-5bc6-4aae-9c51-b8a972bd5978 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:25.656783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f053de9f-5bc6-4aae-9c51-b8a972bd5978 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:25.657225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f053de9f-5bc6-4aae-9c51-b8a972bd5978 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:25.657962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f053de9f-5bc6-4aae-9c51-b8a972bd5978 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:25.658392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f053de9f-5bc6-4aae-9c51-b8a972bd5978 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:25.658827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f053de9f-5bc6-4aae-9c51-b8a972bd5978 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:25.689000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f053de9f-5bc6-4aae-9c51-b8a972bd5978 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045939 Aug 24 10:24:25.689683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 548/957] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:25 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:24:25.713441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a91bf9ff-9ff0-46d0-a9af-c82d9e58addc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:25.715613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a91bf9ff-9ff0-46d0-a9af-c82d9e58addc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:25.721516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a91bf9ff-9ff0-46d0-a9af-c82d9e58addc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:25.721998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a91bf9ff-9ff0-46d0-a9af-c82d9e58addc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:25.722426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a91bf9ff-9ff0-46d0-a9af-c82d9e58addc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:25.723105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a91bf9ff-9ff0-46d0-a9af-c82d9e58addc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:25.723582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a91bf9ff-9ff0-46d0-a9af-c82d9e58addc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:25.724002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a91bf9ff-9ff0-46d0-a9af-c82d9e58addc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:25.753885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a91bf9ff-9ff0-46d0-a9af-c82d9e58addc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044407 Aug 24 10:24:25.754424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 410/958] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:25 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:26.665932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:26.666547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:26.672479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:26.672965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:26.673434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:26.674125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:26.674582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:26.675005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:26.701272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6f4d9212-ac33-410f-9384-198596c79fe5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:26.704740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6f4d9212-ac33-410f-9384-198596c79fe5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:26.711989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f4d9212-ac33-410f-9384-198596c79fe5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:26.712521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f4d9212-ac33-410f-9384-198596c79fe5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:26.712958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f4d9212-ac33-410f-9384-198596c79fe5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:26.713739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f4d9212-ac33-410f-9384-198596c79fe5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:26.714201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f4d9212-ac33-410f-9384-198596c79fe5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:26.714630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f4d9212-ac33-410f-9384-198596c79fe5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:26.731635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:26.745631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6f4d9212-ac33-410f-9384-198596c79fe5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048340 Aug 24 10:24:26.746543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 411/959] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:26 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:24:26.749793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:26.757595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-47113220-04bd-45a6-a4ef-f37deca596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.095604 Aug 24 10:24:26.758164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 549/960] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:26 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:26.766807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-52a3e16a-1eec-4ecc-8e9d-d9715ad49563 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:26.769493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-52a3e16a-1eec-4ecc-8e9d-d9715ad49563 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:26.776595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52a3e16a-1eec-4ecc-8e9d-d9715ad49563 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:26.777079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52a3e16a-1eec-4ecc-8e9d-d9715ad49563 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:26.777543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52a3e16a-1eec-4ecc-8e9d-d9715ad49563 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:26.778230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52a3e16a-1eec-4ecc-8e9d-d9715ad49563 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:26.778678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52a3e16a-1eec-4ecc-8e9d-d9715ad49563 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:26.779102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52a3e16a-1eec-4ecc-8e9d-d9715ad49563 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:26.808404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-52a3e16a-1eec-4ecc-8e9d-d9715ad49563 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046245 Aug 24 10:24:26.808954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 412/961] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:26 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:24:27.758458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-12898597-e683-4b39-a058-6b82bf38d886 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:27.760634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-12898597-e683-4b39-a058-6b82bf38d886 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:27.767722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12898597-e683-4b39-a058-6b82bf38d886 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:27.768243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12898597-e683-4b39-a058-6b82bf38d886 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:27.768684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12898597-e683-4b39-a058-6b82bf38d886 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:27.769387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12898597-e683-4b39-a058-6b82bf38d886 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:27.769844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12898597-e683-4b39-a058-6b82bf38d886 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:27.770272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12898597-e683-4b39-a058-6b82bf38d886 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:27.778535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:27.779149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:27.786572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:27.787063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:27.787499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:27.788237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:27.788695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:27.789120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:27.800464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-12898597-e683-4b39-a058-6b82bf38d886 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045926 Aug 24 10:24:27.801027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 550/962] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:27 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:27.820715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a5668e52-abfa-4f54-9ab1-f52a93162adf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:27.822855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a5668e52-abfa-4f54-9ab1-f52a93162adf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:27.829234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5668e52-abfa-4f54-9ab1-f52a93162adf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:27.829726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5668e52-abfa-4f54-9ab1-f52a93162adf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:27.830165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5668e52-abfa-4f54-9ab1-f52a93162adf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:27.830867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5668e52-abfa-4f54-9ab1-f52a93162adf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:27.831356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5668e52-abfa-4f54-9ab1-f52a93162adf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:27.831782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5668e52-abfa-4f54-9ab1-f52a93162adf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:27.850750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:27.858608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d3b01b8c-05cf-480c-809d-60b6e8218760 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083828 Aug 24 10:24:27.859369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 413/963] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:27 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:24:27.862584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a5668e52-abfa-4f54-9ab1-f52a93162adf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045593 Aug 24 10:24:27.863133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 551/964] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:27 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:24:28.812928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d8477cca-43ef-462c-92a0-c1a5be6dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:28.815071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d8477cca-43ef-462c-92a0-c1a5be6dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:28.821406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8477cca-43ef-462c-92a0-c1a5be6dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:28.821882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8477cca-43ef-462c-92a0-c1a5be6dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:28.822314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8477cca-43ef-462c-92a0-c1a5be6dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:28.822992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8477cca-43ef-462c-92a0-c1a5be6dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:28.823440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8477cca-43ef-462c-92a0-c1a5be6dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:28.823854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8477cca-43ef-462c-92a0-c1a5be6dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:28.854034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d8477cca-43ef-462c-92a0-c1a5be6dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045095 Aug 24 10:24:28.854585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 414/965] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:28 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:28.875484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a1504f28-e903-464f-aa2f-2133d95da12c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:28.877754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a1504f28-e903-464f-aa2f-2133d95da12c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:28.878753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:28.879393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:28.884330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1504f28-e903-464f-aa2f-2133d95da12c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:28.884822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1504f28-e903-464f-aa2f-2133d95da12c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:28.885264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1504f28-e903-464f-aa2f-2133d95da12c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:28.885574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:28.885997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1504f28-e903-464f-aa2f-2133d95da12c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:28.886100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:28.886457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1504f28-e903-464f-aa2f-2133d95da12c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:28.886565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:28.886887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1504f28-e903-464f-aa2f-2133d95da12c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:28.887131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:28.887575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:28.887994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:28.916932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a1504f28-e903-464f-aa2f-2133d95da12c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046053 Aug 24 10:24:28.918615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 552/966] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:28 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:28.949413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:28.957946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2a0cc459-f4a7-4ba9-accc-f0dc5932f4f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082609 Aug 24 10:24:28.958611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 415/967] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:28 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:24:29.866011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-04a2bb72-51ff-4eb3-8bc7-e572ee55dcb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:29.868162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-04a2bb72-51ff-4eb3-8bc7-e572ee55dcb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:29.874097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04a2bb72-51ff-4eb3-8bc7-e572ee55dcb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:29.874574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04a2bb72-51ff-4eb3-8bc7-e572ee55dcb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:29.875011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04a2bb72-51ff-4eb3-8bc7-e572ee55dcb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:29.875699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04a2bb72-51ff-4eb3-8bc7-e572ee55dcb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:29.876275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04a2bb72-51ff-4eb3-8bc7-e572ee55dcb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:29.876596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04a2bb72-51ff-4eb3-8bc7-e572ee55dcb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:29.906339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-04a2bb72-51ff-4eb3-8bc7-e572ee55dcb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043505 Aug 24 10:24:29.906881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 553/968] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:29 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:29.929129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ae1258cc-2d09-4f57-9c88-aa55b798d8f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:29.931352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ae1258cc-2d09-4f57-9c88-aa55b798d8f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:29.937999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae1258cc-2d09-4f57-9c88-aa55b798d8f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:29.938480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae1258cc-2d09-4f57-9c88-aa55b798d8f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:29.938920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae1258cc-2d09-4f57-9c88-aa55b798d8f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:29.939612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae1258cc-2d09-4f57-9c88-aa55b798d8f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:29.940093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae1258cc-2d09-4f57-9c88-aa55b798d8f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:29.940524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae1258cc-2d09-4f57-9c88-aa55b798d8f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:29.970057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ae1258cc-2d09-4f57-9c88-aa55b798d8f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044940 Aug 24 10:24:29.970603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 416/969] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:29 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:29.981585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:29.982293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:29.989753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:29.990320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:29.990852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:29.991666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:29.992230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:29.992723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:30.053891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:30.061904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c547b330-0c6f-4336-a1d2-187e8c02e981 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084718 Aug 24 10:24:30.062465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 554/970] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:29 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:24:30.917909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bd674f09-e4ff-45f4-afaa-78a831a1c81f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:30.920035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bd674f09-e4ff-45f4-afaa-78a831a1c81f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:30.926384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd674f09-e4ff-45f4-afaa-78a831a1c81f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:30.926858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd674f09-e4ff-45f4-afaa-78a831a1c81f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:30.927288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd674f09-e4ff-45f4-afaa-78a831a1c81f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:30.927961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd674f09-e4ff-45f4-afaa-78a831a1c81f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:30.928433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd674f09-e4ff-45f4-afaa-78a831a1c81f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:30.928850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd674f09-e4ff-45f4-afaa-78a831a1c81f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:30.959051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bd674f09-e4ff-45f4-afaa-78a831a1c81f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045068 Aug 24 10:24:30.959703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 417/971] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:30 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:30.982683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-af3239ab-18cc-47ae-968d-e52999dff924 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:30.985471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-af3239ab-18cc-47ae-968d-e52999dff924 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:30.991770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af3239ab-18cc-47ae-968d-e52999dff924 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:30.992286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af3239ab-18cc-47ae-968d-e52999dff924 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:30.992716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af3239ab-18cc-47ae-968d-e52999dff924 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:30.993400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af3239ab-18cc-47ae-968d-e52999dff924 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:30.993859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af3239ab-18cc-47ae-968d-e52999dff924 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:30.994283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af3239ab-18cc-47ae-968d-e52999dff924 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:31.023139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-af3239ab-18cc-47ae-968d-e52999dff924 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044235 Aug 24 10:24:31.023604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 555/972] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:30 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:31.082325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:31.083194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:31.089172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:31.089656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:31.090092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:31.090777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:31.091229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:31.091650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:31.152177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:31.160374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8924f1d1-8057-460a-8188-a4c1bdb949d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081885 Aug 24 10:24:31.160934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 418/973] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:31 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:31.971095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cb07719b-6f07-4f66-a149-bdbece69c9b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:31.973243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cb07719b-6f07-4f66-a149-bdbece69c9b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:31.979174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb07719b-6f07-4f66-a149-bdbece69c9b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:31.979659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb07719b-6f07-4f66-a149-bdbece69c9b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:31.980124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb07719b-6f07-4f66-a149-bdbece69c9b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:31.980819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb07719b-6f07-4f66-a149-bdbece69c9b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:31.981275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb07719b-6f07-4f66-a149-bdbece69c9b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:31.981702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb07719b-6f07-4f66-a149-bdbece69c9b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:31.998159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:32.011691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cb07719b-6f07-4f66-a149-bdbece69c9b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044886 Aug 24 10:24:32.012263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 556/974] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:31 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:32.035052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d762d7da-254b-4baf-9808-f2db291c335a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:32.037214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d762d7da-254b-4baf-9808-f2db291c335a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:32.043627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d762d7da-254b-4baf-9808-f2db291c335a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:32.044137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d762d7da-254b-4baf-9808-f2db291c335a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:32.044569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d762d7da-254b-4baf-9808-f2db291c335a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:32.045254 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d762d7da-254b-4baf-9808-f2db291c335a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:32.045703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d762d7da-254b-4baf-9808-f2db291c335a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:32.046119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d762d7da-254b-4baf-9808-f2db291c335a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:32.074944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d762d7da-254b-4baf-9808-f2db291c335a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043885 Aug 24 10:24:32.075483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 419/975] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:32 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:32.181578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:32.182193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:32.188612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:32.189098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:32.189530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:32.190242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:32.190703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:32.191129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:32.255514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:32.263440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0700f8ee-92c0-4c5d-9c2f-1d19b9eda8a5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085809 Aug 24 10:24:32.264003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 557/976] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:32 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:33.024832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-459c37b6-2179-4ef0-9759-c8c1f6e90bee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:33.026934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-459c37b6-2179-4ef0-9759-c8c1f6e90bee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:33.033248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-459c37b6-2179-4ef0-9759-c8c1f6e90bee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:33.033737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-459c37b6-2179-4ef0-9759-c8c1f6e90bee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:33.034175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-459c37b6-2179-4ef0-9759-c8c1f6e90bee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:33.034877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-459c37b6-2179-4ef0-9759-c8c1f6e90bee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:33.035336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-459c37b6-2179-4ef0-9759-c8c1f6e90bee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:33.036001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-459c37b6-2179-4ef0-9759-c8c1f6e90bee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:33.067278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-459c37b6-2179-4ef0-9759-c8c1f6e90bee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046522 Aug 24 10:24:33.067842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 420/977] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:33 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:33.087923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dd389124-1986-4640-a2a4-f9207ea1232f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:33.090145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dd389124-1986-4640-a2a4-f9207ea1232f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:33.096594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd389124-1986-4640-a2a4-f9207ea1232f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:33.097080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd389124-1986-4640-a2a4-f9207ea1232f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:33.097522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd389124-1986-4640-a2a4-f9207ea1232f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:33.098215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd389124-1986-4640-a2a4-f9207ea1232f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:33.098680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd389124-1986-4640-a2a4-f9207ea1232f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:33.099147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd389124-1986-4640-a2a4-f9207ea1232f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:33.128965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dd389124-1986-4640-a2a4-f9207ea1232f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044944 Aug 24 10:24:33.129525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 558/978] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:33 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:33.284837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:33.285359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:33.291522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:33.291997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:33.292460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:33.293142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:33.293583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:33.293994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:33.368524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:33.378673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8e739486-eef4-490a-b469-8ec4826b0695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.097470 Aug 24 10:24:33.379640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 421/979] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:33 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:34.079474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f29f9ed0-2a76-48a6-93f5-48be06b8202a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:34.081589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f29f9ed0-2a76-48a6-93f5-48be06b8202a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:34.087655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f29f9ed0-2a76-48a6-93f5-48be06b8202a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:34.088178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f29f9ed0-2a76-48a6-93f5-48be06b8202a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:34.088778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f29f9ed0-2a76-48a6-93f5-48be06b8202a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:34.089288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f29f9ed0-2a76-48a6-93f5-48be06b8202a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:34.089745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f29f9ed0-2a76-48a6-93f5-48be06b8202a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:34.090165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f29f9ed0-2a76-48a6-93f5-48be06b8202a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:34.118511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f29f9ed0-2a76-48a6-93f5-48be06b8202a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043576 Aug 24 10:24:34.119047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 559/980] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:34 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:34.141967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e367349e-078c-409f-a07d-a6fe5c974f6f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:34.144245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e367349e-078c-409f-a07d-a6fe5c974f6f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:34.150635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e367349e-078c-409f-a07d-a6fe5c974f6f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:34.151118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e367349e-078c-409f-a07d-a6fe5c974f6f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:34.151564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e367349e-078c-409f-a07d-a6fe5c974f6f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:34.152288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e367349e-078c-409f-a07d-a6fe5c974f6f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:34.152750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e367349e-078c-409f-a07d-a6fe5c974f6f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:34.153180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e367349e-078c-409f-a07d-a6fe5c974f6f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:34.181582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e367349e-078c-409f-a07d-a6fe5c974f6f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044425 Aug 24 10:24:34.182123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 422/981] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:34 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:34.399885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:34.400487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:34.406319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:34.406802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:34.407240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:34.407922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:34.408400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:34.408825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:34.470564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:34.478195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e5c6f296-10ff-46e3-a2d3-3992d59f495d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082269 Aug 24 10:24:34.478749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 560/982] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:34 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:35.130879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ed0c7406-56c0-44fb-995f-557379a0a741 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:35.133052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ed0c7406-56c0-44fb-995f-557379a0a741 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:35.138951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed0c7406-56c0-44fb-995f-557379a0a741 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:35.139434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed0c7406-56c0-44fb-995f-557379a0a741 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:35.139873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed0c7406-56c0-44fb-995f-557379a0a741 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:35.140583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed0c7406-56c0-44fb-995f-557379a0a741 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:35.141037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed0c7406-56c0-44fb-995f-557379a0a741 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:35.141460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed0c7406-56c0-44fb-995f-557379a0a741 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:35.172341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ed0c7406-56c0-44fb-995f-557379a0a741 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045389 Aug 24 10:24:35.172936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 423/983] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:35 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:35.193691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-34307875-2645-4404-8d33-dc8a34d765f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:35.195864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-34307875-2645-4404-8d33-dc8a34d765f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:35.202247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34307875-2645-4404-8d33-dc8a34d765f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:35.202733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34307875-2645-4404-8d33-dc8a34d765f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:35.203176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34307875-2645-4404-8d33-dc8a34d765f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:35.203869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34307875-2645-4404-8d33-dc8a34d765f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:35.204356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34307875-2645-4404-8d33-dc8a34d765f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:35.204791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34307875-2645-4404-8d33-dc8a34d765f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:35.235917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-34307875-2645-4404-8d33-dc8a34d765f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046143 Aug 24 10:24:35.236487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 561/984] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:35 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:35.241825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:24:35.241966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:24:35.499112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:35.499715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:35.505594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:35.506077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:35.506501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:35.507179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:35.507623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:35.508037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:35.569326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:35.577477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b476d26c-ac71-4781-9bf1-21f60f2fe97b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081749 Aug 24 10:24:35.578029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 424/985] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:35 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:36.184899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6bdd770b-4cdf-46e7-907d-4a1d30e3554d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:36.187743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6bdd770b-4cdf-46e7-907d-4a1d30e3554d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:36.193681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6bdd770b-4cdf-46e7-907d-4a1d30e3554d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:36.194157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6bdd770b-4cdf-46e7-907d-4a1d30e3554d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:36.194725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6bdd770b-4cdf-46e7-907d-4a1d30e3554d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:36.195271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6bdd770b-4cdf-46e7-907d-4a1d30e3554d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:36.195725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6bdd770b-4cdf-46e7-907d-4a1d30e3554d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:36.196208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6bdd770b-4cdf-46e7-907d-4a1d30e3554d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:36.224752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6bdd770b-4cdf-46e7-907d-4a1d30e3554d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044005 Aug 24 10:24:36.225298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 562/986] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:36 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:36.248440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1d6bd3ba-19ec-46bf-9946-daefd571c43e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:36.251331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1d6bd3ba-19ec-46bf-9946-daefd571c43e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:36.257791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d6bd3ba-19ec-46bf-9946-daefd571c43e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:36.258306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d6bd3ba-19ec-46bf-9946-daefd571c43e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:36.258742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d6bd3ba-19ec-46bf-9946-daefd571c43e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:36.259431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d6bd3ba-19ec-46bf-9946-daefd571c43e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:36.259889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d6bd3ba-19ec-46bf-9946-daefd571c43e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:36.260347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d6bd3ba-19ec-46bf-9946-daefd571c43e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:36.289171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1d6bd3ba-19ec-46bf-9946-daefd571c43e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045034 Aug 24 10:24:36.289721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 425/987] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:36 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:36.598233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:36.598753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:36.604607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:36.605082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:36.605516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:36.606192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:36.606640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:36.607061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:36.668277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:36.676856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c4d4e89a-2519-4901-82d3-d800437c8811 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082562 Aug 24 10:24:36.677421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 563/988] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:36 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:37.236984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dbc12603-4dac-491d-9737-d948ca545a16 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:37.239172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dbc12603-4dac-491d-9737-d948ca545a16 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:37.245151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dbc12603-4dac-491d-9737-d948ca545a16 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:37.245639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dbc12603-4dac-491d-9737-d948ca545a16 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:37.246095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dbc12603-4dac-491d-9737-d948ca545a16 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:37.246771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dbc12603-4dac-491d-9737-d948ca545a16 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:37.247230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dbc12603-4dac-491d-9737-d948ca545a16 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:37.247655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dbc12603-4dac-491d-9737-d948ca545a16 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:37.280392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dbc12603-4dac-491d-9737-d948ca545a16 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047466 Aug 24 10:24:37.281045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 426/989] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:37 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:37.301243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-47f4894a-8621-4831-a668-a0795da95f35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:37.303729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-47f4894a-8621-4831-a668-a0795da95f35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:37.310101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47f4894a-8621-4831-a668-a0795da95f35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:37.310589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47f4894a-8621-4831-a668-a0795da95f35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:37.311526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47f4894a-8621-4831-a668-a0795da95f35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:37.312250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47f4894a-8621-4831-a668-a0795da95f35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:37.312708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47f4894a-8621-4831-a668-a0795da95f35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:37.313138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47f4894a-8621-4831-a668-a0795da95f35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:37.342375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-47f4894a-8621-4831-a668-a0795da95f35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044999 Aug 24 10:24:37.342914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 564/990] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:37 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:37.697119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:37.697740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:37.703646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:37.704157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:37.704591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:37.705271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:37.705724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:37.706145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:37.766657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:37.774580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e23815f7-c881-4613-9512-97b3845c97b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081325 Aug 24 10:24:37.775139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 427/991] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:37 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:38.292330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a321e543-e741-459c-8016-7373cad812c9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:38.294441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a321e543-e741-459c-8016-7373cad812c9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:38.301003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a321e543-e741-459c-8016-7373cad812c9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:38.301490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a321e543-e741-459c-8016-7373cad812c9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:38.302678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a321e543-e741-459c-8016-7373cad812c9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:38.302678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a321e543-e741-459c-8016-7373cad812c9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:38.303054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a321e543-e741-459c-8016-7373cad812c9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:38.303477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a321e543-e741-459c-8016-7373cad812c9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:38.334938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a321e543-e741-459c-8016-7373cad812c9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046517 Aug 24 10:24:38.335509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 565/992] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:38 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:24:38.354437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a481f473-0741-4e0b-a931-ec5258ae1ee6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:38.356737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a481f473-0741-4e0b-a931-ec5258ae1ee6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:38.363578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a481f473-0741-4e0b-a931-ec5258ae1ee6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:38.364236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a481f473-0741-4e0b-a931-ec5258ae1ee6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:38.364679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a481f473-0741-4e0b-a931-ec5258ae1ee6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:38.365375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a481f473-0741-4e0b-a931-ec5258ae1ee6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:38.365826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a481f473-0741-4e0b-a931-ec5258ae1ee6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:38.366242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a481f473-0741-4e0b-a931-ec5258ae1ee6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:38.370071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:38.395877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a481f473-0741-4e0b-a931-ec5258ae1ee6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045344 Aug 24 10:24:38.396450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 428/993] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:38 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:38.794762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:38.796020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:38.801891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:38.802372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:38.802815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:38.803500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:38.803955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:38.804416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:38.864884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:38.872591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c6a8ce3b-5b7e-4209-8b05-1c256e4bdf5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081735 Aug 24 10:24:38.873145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 566/994] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:38 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:39.348727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9b7f054f-39b4-4be3-b789-f4d33a97ee6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:39.351048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9b7f054f-39b4-4be3-b789-f4d33a97ee6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:39.358747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7f054f-39b4-4be3-b789-f4d33a97ee6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:39.359354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7f054f-39b4-4be3-b789-f4d33a97ee6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:39.359829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7f054f-39b4-4be3-b789-f4d33a97ee6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:39.360703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7f054f-39b4-4be3-b789-f4d33a97ee6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:39.361226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7f054f-39b4-4be3-b789-f4d33a97ee6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:39.361715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7f054f-39b4-4be3-b789-f4d33a97ee6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:39.392272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9b7f054f-39b4-4be3-b789-f4d33a97ee6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047758 Aug 24 10:24:39.392835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 429/995] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:39 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:39.409719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eba0fbb4-0736-4800-82ab-d28a4603e358 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:39.411776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eba0fbb4-0736-4800-82ab-d28a4603e358 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:39.418337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba0fbb4-0736-4800-82ab-d28a4603e358 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:39.419330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba0fbb4-0736-4800-82ab-d28a4603e358 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:39.424933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba0fbb4-0736-4800-82ab-d28a4603e358 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:39.425661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba0fbb4-0736-4800-82ab-d28a4603e358 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:39.426119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba0fbb4-0736-4800-82ab-d28a4603e358 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:39.426547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba0fbb4-0736-4800-82ab-d28a4603e358 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:39.455217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eba0fbb4-0736-4800-82ab-d28a4603e358 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050790 Aug 24 10:24:39.455763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 567/996] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:39 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:39.893796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:39.894447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:39.900479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:39.900961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:39.901399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:39.902083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:39.902540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:39.902964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:39.963752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:39.972557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4aa50474-7a58-475a-866a-d9f7d9be3216 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083346 Aug 24 10:24:39.973108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 430/997] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:39 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:24:40.403659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5af71ac9-e49b-4ec3-b733-d4faab9d0d53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:40.406229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5af71ac9-e49b-4ec3-b733-d4faab9d0d53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:40.412278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af71ac9-e49b-4ec3-b733-d4faab9d0d53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:40.412885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af71ac9-e49b-4ec3-b733-d4faab9d0d53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:40.413237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af71ac9-e49b-4ec3-b733-d4faab9d0d53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:40.413943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af71ac9-e49b-4ec3-b733-d4faab9d0d53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:40.414401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af71ac9-e49b-4ec3-b733-d4faab9d0d53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:40.414834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af71ac9-e49b-4ec3-b733-d4faab9d0d53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:40.444864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5af71ac9-e49b-4ec3-b733-d4faab9d0d53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045250 Aug 24 10:24:40.445484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 568/998] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:40 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:40.466886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ba5265d7-b2fb-44c6-a76e-544289f29ca3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:40.469584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ba5265d7-b2fb-44c6-a76e-544289f29ca3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:40.475942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba5265d7-b2fb-44c6-a76e-544289f29ca3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:40.476435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba5265d7-b2fb-44c6-a76e-544289f29ca3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:40.476874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba5265d7-b2fb-44c6-a76e-544289f29ca3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:40.477561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba5265d7-b2fb-44c6-a76e-544289f29ca3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:40.478019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba5265d7-b2fb-44c6-a76e-544289f29ca3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:40.478446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba5265d7-b2fb-44c6-a76e-544289f29ca3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:40.507779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ba5265d7-b2fb-44c6-a76e-544289f29ca3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044910 Aug 24 10:24:40.508363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 431/999] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:40 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:40.993278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:40.993933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:41.000597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:41.001085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:41.001527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:41.002222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:41.002678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:41.003109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:41.081583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:41.089700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-877f7411-e9dd-46ca-9b9a-a03cb1e2c0f3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.100411 Aug 24 10:24:41.090263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 569/1000] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:40 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:24:41.457811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-61fe1b98-ffe2-4c30-bc99-7019e60a23e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:41.459985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-61fe1b98-ffe2-4c30-bc99-7019e60a23e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:41.465993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fe1b98-ffe2-4c30-bc99-7019e60a23e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:41.466472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fe1b98-ffe2-4c30-bc99-7019e60a23e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:41.466907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fe1b98-ffe2-4c30-bc99-7019e60a23e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:41.467602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fe1b98-ffe2-4c30-bc99-7019e60a23e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:41.468065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fe1b98-ffe2-4c30-bc99-7019e60a23e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:41.468509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fe1b98-ffe2-4c30-bc99-7019e60a23e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:41.497682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-61fe1b98-ffe2-4c30-bc99-7019e60a23e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044361 Aug 24 10:24:41.498890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 432/1001] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:41 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:41.520084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-548db17f-34ef-49a0-8afd-2e89144dce6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:41.522238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-548db17f-34ef-49a0-8afd-2e89144dce6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:41.528557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-548db17f-34ef-49a0-8afd-2e89144dce6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:41.529079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-548db17f-34ef-49a0-8afd-2e89144dce6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:41.529524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-548db17f-34ef-49a0-8afd-2e89144dce6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:41.530226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-548db17f-34ef-49a0-8afd-2e89144dce6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:41.530684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-548db17f-34ef-49a0-8afd-2e89144dce6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:41.531113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-548db17f-34ef-49a0-8afd-2e89144dce6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:41.559766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-548db17f-34ef-49a0-8afd-2e89144dce6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043728 Aug 24 10:24:41.560336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 570/1002] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:41 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:42.109942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:42.110463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:42.116442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:42.116918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:42.117345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:42.118022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:42.118470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:42.118888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:42.182061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:42.189682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-50e4492d-33ab-4784-81ba-e0d7af051ed9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083790 Aug 24 10:24:42.190229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 433/1003] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:42 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:42.510468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1c838df4-ed39-4be3-bc44-7b037c510e01 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:42.512622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1c838df4-ed39-4be3-bc44-7b037c510e01 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:42.519132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c838df4-ed39-4be3-bc44-7b037c510e01 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:42.519617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c838df4-ed39-4be3-bc44-7b037c510e01 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:42.520074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c838df4-ed39-4be3-bc44-7b037c510e01 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:42.520774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c838df4-ed39-4be3-bc44-7b037c510e01 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:42.521307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c838df4-ed39-4be3-bc44-7b037c510e01 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:42.521645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c838df4-ed39-4be3-bc44-7b037c510e01 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:42.550499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1c838df4-ed39-4be3-bc44-7b037c510e01 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043939 Aug 24 10:24:42.551046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 571/1004] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:42 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:42.572735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-31722cb4-eaba-4848-83b4-c60a2629a57d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:42.574888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-31722cb4-eaba-4848-83b4-c60a2629a57d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:42.581913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31722cb4-eaba-4848-83b4-c60a2629a57d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:42.582652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31722cb4-eaba-4848-83b4-c60a2629a57d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:42.583087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31722cb4-eaba-4848-83b4-c60a2629a57d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:42.583776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31722cb4-eaba-4848-83b4-c60a2629a57d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:42.584274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31722cb4-eaba-4848-83b4-c60a2629a57d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:42.584699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31722cb4-eaba-4848-83b4-c60a2629a57d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:42.613207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-31722cb4-eaba-4848-83b4-c60a2629a57d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044507 Aug 24 10:24:42.613755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 434/1005] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:42 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:43.211106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:43.211724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:43.218094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:43.218578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:43.219046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:43.219736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:43.220230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:43.220656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:43.280358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:43.287921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6629071d-1138-4e90-b58f-23f476416064 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081052 Aug 24 10:24:43.288524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 572/1006] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:43 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:43.562877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-de631552-5a1c-4d3f-9476-6cadfb8bdb5a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:43.565122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-de631552-5a1c-4d3f-9476-6cadfb8bdb5a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:43.571191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de631552-5a1c-4d3f-9476-6cadfb8bdb5a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:43.571681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de631552-5a1c-4d3f-9476-6cadfb8bdb5a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:43.572143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de631552-5a1c-4d3f-9476-6cadfb8bdb5a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:43.572838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de631552-5a1c-4d3f-9476-6cadfb8bdb5a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:43.573391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de631552-5a1c-4d3f-9476-6cadfb8bdb5a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:43.573705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de631552-5a1c-4d3f-9476-6cadfb8bdb5a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:43.604008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-de631552-5a1c-4d3f-9476-6cadfb8bdb5a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045110 Aug 24 10:24:43.604612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 435/1007] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:43 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:43.624758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e46d9f1f-20ef-42bc-afab-c50d26ca3264 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:43.627000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e46d9f1f-20ef-42bc-afab-c50d26ca3264 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:43.633384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d9f1f-20ef-42bc-afab-c50d26ca3264 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:43.633870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d9f1f-20ef-42bc-afab-c50d26ca3264 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:43.634366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d9f1f-20ef-42bc-afab-c50d26ca3264 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:43.635065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d9f1f-20ef-42bc-afab-c50d26ca3264 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:43.635875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d9f1f-20ef-42bc-afab-c50d26ca3264 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:43.636337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e46d9f1f-20ef-42bc-afab-c50d26ca3264 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:43.640619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:43.666337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e46d9f1f-20ef-42bc-afab-c50d26ca3264 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045442 Aug 24 10:24:43.666883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 573/1008] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:43 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:44.309267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:44.309797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:44.316632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:44.317122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:44.317564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:44.318252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:44.318710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:44.319652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:44.382331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:44.389927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b2068fee-ba32-4c0f-b33c-b1bb481f04ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084709 Aug 24 10:24:44.390489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 436/1009] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:44 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:44.616788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0e7f1561-52af-4840-ab31-f04c2ab68cda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:44.618840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0e7f1561-52af-4840-ab31-f04c2ab68cda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:44.625810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e7f1561-52af-4840-ab31-f04c2ab68cda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:44.626406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e7f1561-52af-4840-ab31-f04c2ab68cda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:44.626725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e7f1561-52af-4840-ab31-f04c2ab68cda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:44.627422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e7f1561-52af-4840-ab31-f04c2ab68cda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:44.627911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e7f1561-52af-4840-ab31-f04c2ab68cda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:44.628380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e7f1561-52af-4840-ab31-f04c2ab68cda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:44.657719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0e7f1561-52af-4840-ab31-f04c2ab68cda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045089 Aug 24 10:24:44.658255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 574/1010] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:44 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:44.678550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c433b812-1ea7-42c6-bb02-104c0e791c29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:44.680776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c433b812-1ea7-42c6-bb02-104c0e791c29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:44.687075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b812-1ea7-42c6-bb02-104c0e791c29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:44.687553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b812-1ea7-42c6-bb02-104c0e791c29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:44.687986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b812-1ea7-42c6-bb02-104c0e791c29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:44.688694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b812-1ea7-42c6-bb02-104c0e791c29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:44.689151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b812-1ea7-42c6-bb02-104c0e791c29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:44.689572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b812-1ea7-42c6-bb02-104c0e791c29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:44.719260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c433b812-1ea7-42c6-bb02-104c0e791c29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044627 Aug 24 10:24:44.719816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 437/1011] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:44 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:45.410282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:45.410923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:45.416825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:45.417307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:45.417749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:45.418444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:45.418900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:45.419328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:45.480314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:45.488596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0239a1db-45fa-4b30-b716-1b1d8a64544a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082276 Aug 24 10:24:45.489154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 575/1012] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:45 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:45.670823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d2475e27-1cd6-418e-9635-0f92dd1d4b8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:45.672743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d2475e27-1cd6-418e-9635-0f92dd1d4b8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:45.678731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2475e27-1cd6-418e-9635-0f92dd1d4b8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:45.679295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2475e27-1cd6-418e-9635-0f92dd1d4b8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:45.679634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2475e27-1cd6-418e-9635-0f92dd1d4b8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:45.680363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2475e27-1cd6-418e-9635-0f92dd1d4b8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:45.680826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2475e27-1cd6-418e-9635-0f92dd1d4b8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:45.681254 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2475e27-1cd6-418e-9635-0f92dd1d4b8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:45.710384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d2475e27-1cd6-418e-9635-0f92dd1d4b8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044361 Aug 24 10:24:45.710935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 438/1013] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:45 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:45.731097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0960e8b4-a9fe-4e1f-a45c-1c743fdd32d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:45.733917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0960e8b4-a9fe-4e1f-a45c-1c743fdd32d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:45.740475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0960e8b4-a9fe-4e1f-a45c-1c743fdd32d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:45.740955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0960e8b4-a9fe-4e1f-a45c-1c743fdd32d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:45.741390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0960e8b4-a9fe-4e1f-a45c-1c743fdd32d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:45.742072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0960e8b4-a9fe-4e1f-a45c-1c743fdd32d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:45.742522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0960e8b4-a9fe-4e1f-a45c-1c743fdd32d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:45.742942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0960e8b4-a9fe-4e1f-a45c-1c743fdd32d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:45.771462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0960e8b4-a9fe-4e1f-a45c-1c743fdd32d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044183 Aug 24 10:24:45.772015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 576/1014] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:45 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:46.509457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:46.510072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:46.516646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:46.517135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:46.517573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:46.518290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:46.518750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:46.519174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:46.579559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:46.587692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a939e843-9023-4c4e-8264-bbfb45054428 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082131 Aug 24 10:24:46.588302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 439/1015] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:46 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:46.722747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4f7c7e10-23dc-45d3-a8be-4b6917ab8c0f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:46.724920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4f7c7e10-23dc-45d3-a8be-4b6917ab8c0f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:46.731039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f7c7e10-23dc-45d3-a8be-4b6917ab8c0f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:46.731525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f7c7e10-23dc-45d3-a8be-4b6917ab8c0f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:46.732113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f7c7e10-23dc-45d3-a8be-4b6917ab8c0f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:46.733246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f7c7e10-23dc-45d3-a8be-4b6917ab8c0f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:46.733701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f7c7e10-23dc-45d3-a8be-4b6917ab8c0f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:46.734132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4f7c7e10-23dc-45d3-a8be-4b6917ab8c0f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:46.764396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4f7c7e10-23dc-45d3-a8be-4b6917ab8c0f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045443 Aug 24 10:24:46.764875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 577/1016] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:46 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:46.782781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-abcc8b52-8673-4a39-be3c-f0c331b6d801 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:46.785082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-abcc8b52-8673-4a39-be3c-f0c331b6d801 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:46.791635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abcc8b52-8673-4a39-be3c-f0c331b6d801 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:46.792151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abcc8b52-8673-4a39-be3c-f0c331b6d801 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:46.792588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abcc8b52-8673-4a39-be3c-f0c331b6d801 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:46.793282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abcc8b52-8673-4a39-be3c-f0c331b6d801 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:46.793737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abcc8b52-8673-4a39-be3c-f0c331b6d801 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:46.794162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abcc8b52-8673-4a39-be3c-f0c331b6d801 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:46.823849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-abcc8b52-8673-4a39-be3c-f0c331b6d801 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045008 Aug 24 10:24:46.824428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 440/1017] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:46 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:46.973599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:24:46.973780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:24:47.608608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:47.609233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:47.615001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:47.615477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:47.615908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:47.616610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:47.617062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:47.617481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:47.676907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:47.684422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-24b7ceb4-d9dc-48ee-8c8c-c4c9bec564bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.079958 Aug 24 10:24:47.684963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 578/1018] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:47 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:47.776529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ed7c4a11-0a1f-478b-8b0d-734cf7319799 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:47.778670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ed7c4a11-0a1f-478b-8b0d-734cf7319799 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:47.780674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=85 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:47.786343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed7c4a11-0a1f-478b-8b0d-734cf7319799 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:47.786955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed7c4a11-0a1f-478b-8b0d-734cf7319799 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:47.787237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed7c4a11-0a1f-478b-8b0d-734cf7319799 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:47.787929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed7c4a11-0a1f-478b-8b0d-734cf7319799 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:47.788403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed7c4a11-0a1f-478b-8b0d-734cf7319799 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:47.788822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed7c4a11-0a1f-478b-8b0d-734cf7319799 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:47.818569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ed7c4a11-0a1f-478b-8b0d-734cf7319799 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045937 Aug 24 10:24:47.819107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 441/1019] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:47 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:47.836701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-20f5e426-2b5f-4237-92af-f87a4d885c68 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:47.838892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-20f5e426-2b5f-4237-92af-f87a4d885c68 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:47.845180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f5e426-2b5f-4237-92af-f87a4d885c68 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:47.845665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f5e426-2b5f-4237-92af-f87a4d885c68 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:47.846108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f5e426-2b5f-4237-92af-f87a4d885c68 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:47.846804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f5e426-2b5f-4237-92af-f87a4d885c68 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:47.847263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f5e426-2b5f-4237-92af-f87a4d885c68 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:47.847696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f5e426-2b5f-4237-92af-f87a4d885c68 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:47.876382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-20f5e426-2b5f-4237-92af-f87a4d885c68 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043514 Aug 24 10:24:47.876846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 579/1020] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:47 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:48.705420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:48.706032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:48.712272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:48.712760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:48.713197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:48.713890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:48.714347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:48.714774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:48.775382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:48.783824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-896e8435-493f-446e-8ad7-b8db232f40bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082341 Aug 24 10:24:48.784413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 442/1021] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:48 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:48.831305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4d04eedb-2281-421f-9a9a-2ce984faa955 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:48.833724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4d04eedb-2281-421f-9a9a-2ce984faa955 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:48.836667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=83 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:48.846424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d04eedb-2281-421f-9a9a-2ce984faa955 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:48.847407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d04eedb-2281-421f-9a9a-2ce984faa955 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:48.847407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d04eedb-2281-421f-9a9a-2ce984faa955 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:48.848111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d04eedb-2281-421f-9a9a-2ce984faa955 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:48.848567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d04eedb-2281-421f-9a9a-2ce984faa955 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:48.848989 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d04eedb-2281-421f-9a9a-2ce984faa955 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:48.877839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4d04eedb-2281-421f-9a9a-2ce984faa955 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050513 Aug 24 10:24:48.878376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 580/1022] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:48 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:48.889201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d88c86f8-9a9a-4996-859e-98338c88d3c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:48.891430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d88c86f8-9a9a-4996-859e-98338c88d3c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:48.898210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d88c86f8-9a9a-4996-859e-98338c88d3c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:48.898691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d88c86f8-9a9a-4996-859e-98338c88d3c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:48.899127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d88c86f8-9a9a-4996-859e-98338c88d3c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:48.899824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d88c86f8-9a9a-4996-859e-98338c88d3c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:48.900304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d88c86f8-9a9a-4996-859e-98338c88d3c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:48.900731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d88c86f8-9a9a-4996-859e-98338c88d3c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:48.930198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d88c86f8-9a9a-4996-859e-98338c88d3c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045659 Aug 24 10:24:48.930753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 443/1023] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:48 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:49.804475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:49.805094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:49.811609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:49.812118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:49.812561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:49.813249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:49.813706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:49.814137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:49.877321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:49.885055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3d9ec01b-c6d8-4a2d-842d-f14353bc5b81 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084582 Aug 24 10:24:49.885611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 581/1024] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:49 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:49.890299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ed9171c5-e14c-4917-ad93-37a47d1daca6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:49.892471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ed9171c5-e14c-4917-ad93-37a47d1daca6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:49.898431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed9171c5-e14c-4917-ad93-37a47d1daca6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:49.898927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed9171c5-e14c-4917-ad93-37a47d1daca6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:49.899348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed9171c5-e14c-4917-ad93-37a47d1daca6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:49.900052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed9171c5-e14c-4917-ad93-37a47d1daca6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:49.900524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed9171c5-e14c-4917-ad93-37a47d1daca6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:49.900951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed9171c5-e14c-4917-ad93-37a47d1daca6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:49.927140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:49.932543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ed9171c5-e14c-4917-ad93-37a47d1daca6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046376 Aug 24 10:24:49.933102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 444/1025] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:49 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:24:49.942435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8e796536-723a-46e0-abfa-98e4a0740436 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:49.944594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8e796536-723a-46e0-abfa-98e4a0740436 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:49.950497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e796536-723a-46e0-abfa-98e4a0740436 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:49.950990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e796536-723a-46e0-abfa-98e4a0740436 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:49.951440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e796536-723a-46e0-abfa-98e4a0740436 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:49.952161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e796536-723a-46e0-abfa-98e4a0740436 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:49.952633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e796536-723a-46e0-abfa-98e4a0740436 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:49.953068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8e796536-723a-46e0-abfa-98e4a0740436 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:49.982302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8e796536-723a-46e0-abfa-98e4a0740436 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043725 Aug 24 10:24:49.982844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 582/1026] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:49 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:50.645699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:24:50.645929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:24:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:24:50.905512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:50.906148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:50.912648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:50.913134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:50.913573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:50.914262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:50.914721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:50.915150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:50.944689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5afea129-e357-43b8-b29c-0e536034d5a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:50.946933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5afea129-e357-43b8-b29c-0e536034d5a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:50.954316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5afea129-e357-43b8-b29c-0e536034d5a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:50.954886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5afea129-e357-43b8-b29c-0e536034d5a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:50.955436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5afea129-e357-43b8-b29c-0e536034d5a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:50.956173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5afea129-e357-43b8-b29c-0e536034d5a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:50.956626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5afea129-e357-43b8-b29c-0e536034d5a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:50.957311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5afea129-e357-43b8-b29c-0e536034d5a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:50.979290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:50.987045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-00265847-84e1-458d-9ac0-d4ea640ad184 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085486 Aug 24 10:24:50.987413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5afea129-e357-43b8-b29c-0e536034d5a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046517 Aug 24 10:24:50.987601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 445/1027] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:50 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:50.989966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 583/1028] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:50 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:24:50.995419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-658cd7c9-89ec-4ec4-b789-8afe7c0da097 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:50.998134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-658cd7c9-89ec-4ec4-b789-8afe7c0da097 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:51.004309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-658cd7c9-89ec-4ec4-b789-8afe7c0da097 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:51.004789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-658cd7c9-89ec-4ec4-b789-8afe7c0da097 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:51.005218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-658cd7c9-89ec-4ec4-b789-8afe7c0da097 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:51.005904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-658cd7c9-89ec-4ec4-b789-8afe7c0da097 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:51.006349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-658cd7c9-89ec-4ec4-b789-8afe7c0da097 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:51.006765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-658cd7c9-89ec-4ec4-b789-8afe7c0da097 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:51.036532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-658cd7c9-89ec-4ec4-b789-8afe7c0da097 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044821 Aug 24 10:24:51.037085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 446/1029] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:50 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:51.999515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b9ff465a-cc7a-44f0-9f00-bab744743cd5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:52.001679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b9ff465a-cc7a-44f0-9f00-bab744743cd5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:52.007749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:52.008264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9ff465a-cc7a-44f0-9f00-bab744743cd5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:52.008409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:52.008648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9ff465a-cc7a-44f0-9f00-bab744743cd5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:52.009086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9ff465a-cc7a-44f0-9f00-bab744743cd5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:52.009813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9ff465a-cc7a-44f0-9f00-bab744743cd5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:52.010272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9ff465a-cc7a-44f0-9f00-bab744743cd5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:52.010698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9ff465a-cc7a-44f0-9f00-bab744743cd5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:52.015065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:52.015618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:52.016065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:52.016764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:52.017213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:52.017630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:52.040370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b9ff465a-cc7a-44f0-9f00-bab744743cd5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045296 Aug 24 10:24:52.041435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 584/1030] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:51 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:52.051227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cdc5b680-7981-4765-b968-49753a2f160b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:52.053473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cdc5b680-7981-4765-b968-49753a2f160b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:52.059544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdc5b680-7981-4765-b968-49753a2f160b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:52.060630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdc5b680-7981-4765-b968-49753a2f160b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:52.061089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdc5b680-7981-4765-b968-49753a2f160b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:52.061802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdc5b680-7981-4765-b968-49753a2f160b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:52.062257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdc5b680-7981-4765-b968-49753a2f160b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:52.062757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdc5b680-7981-4765-b968-49753a2f160b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:52.079681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:24:52.080016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:52.082227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:24:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments (ip 10.222.0.86) !!! Aug 24 10:24:52.088486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3f45827f-12ae-4292-8a35-293ee0cc03f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084526 Aug 24 10:24:52.088913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 447/1031] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:52 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:24:52.096542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cdc5b680-7981-4765-b968-49753a2f160b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049150 Aug 24 10:24:52.097095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 585/1032] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:52 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:53.052608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-75e3242b-0741-4c8c-8ae4-5f09d75db8b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:53.054866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-75e3242b-0741-4c8c-8ae4-5f09d75db8b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:53.061647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e3242b-0741-4c8c-8ae4-5f09d75db8b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:53.062129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e3242b-0741-4c8c-8ae4-5f09d75db8b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:53.062558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e3242b-0741-4c8c-8ae4-5f09d75db8b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:53.063246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e3242b-0741-4c8c-8ae4-5f09d75db8b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:53.063698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e3242b-0741-4c8c-8ae4-5f09d75db8b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:53.064141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e3242b-0741-4c8c-8ae4-5f09d75db8b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:53.093185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-75e3242b-0741-4c8c-8ae4-5f09d75db8b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044595 Aug 24 10:24:53.093729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 448/1033] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:53 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:53.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:53.109823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:53.110290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ba9d7a34-1c7c-42d1-9651-1b567c17b9e8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:53.112168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ba9d7a34-1c7c-42d1-9651-1b567c17b9e8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:53.117510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:53.118018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:53.118453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:53.118584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba9d7a34-1c7c-42d1-9651-1b567c17b9e8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:53.119035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba9d7a34-1c7c-42d1-9651-1b567c17b9e8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:53.119157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:53.119471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba9d7a34-1c7c-42d1-9651-1b567c17b9e8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:53.119601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:53.119994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:53.120186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba9d7a34-1c7c-42d1-9651-1b567c17b9e8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:53.120636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba9d7a34-1c7c-42d1-9651-1b567c17b9e8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:53.121055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba9d7a34-1c7c-42d1-9651-1b567c17b9e8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:53.150861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ba9d7a34-1c7c-42d1-9651-1b567c17b9e8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045606 Aug 24 10:24:53.151409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 586/1034] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:53.181278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:53.189469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-367dd50b-a9b3-4a27-87fd-f4dc12c4a0c9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083989 Aug 24 10:24:53.190042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 449/1035] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:53 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:24:54.105369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3707a603-fcaf-43fd-9348-d744c2072c69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:54.108002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3707a603-fcaf-43fd-9348-d744c2072c69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:54.114045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3707a603-fcaf-43fd-9348-d744c2072c69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:54.114628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3707a603-fcaf-43fd-9348-d744c2072c69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:54.114962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3707a603-fcaf-43fd-9348-d744c2072c69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:54.115657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3707a603-fcaf-43fd-9348-d744c2072c69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:54.116146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3707a603-fcaf-43fd-9348-d744c2072c69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:54.116577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3707a603-fcaf-43fd-9348-d744c2072c69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:54.146230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3707a603-fcaf-43fd-9348-d744c2072c69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044910 Aug 24 10:24:54.146774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 587/1036] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:54 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:54.162751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cef28fe1-a92f-4098-bebd-6496d67ba66d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:54.164938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cef28fe1-a92f-4098-bebd-6496d67ba66d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:54.171892 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cef28fe1-a92f-4098-bebd-6496d67ba66d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:54.172399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cef28fe1-a92f-4098-bebd-6496d67ba66d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:54.172830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cef28fe1-a92f-4098-bebd-6496d67ba66d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:54.173512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cef28fe1-a92f-4098-bebd-6496d67ba66d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:54.173961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cef28fe1-a92f-4098-bebd-6496d67ba66d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:54.174379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cef28fe1-a92f-4098-bebd-6496d67ba66d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:54.203195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cef28fe1-a92f-4098-bebd-6496d67ba66d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044371 Aug 24 10:24:54.204583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 450/1037] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:54 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:54.210283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:54.210919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:54.216826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:54.217303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:54.217735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:54.218413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:54.218863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:54.219286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:54.279043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:54.286843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b722ffce-d989-4d64-a255-abe2fdfd5ef6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080575 Aug 24 10:24:54.287426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 588/1038] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:54 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:55.159631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ce59b685-3d14-4497-b8f1-b5744b58b5d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:55.161206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ce59b685-3d14-4497-b8f1-b5744b58b5d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:55.167072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce59b685-3d14-4497-b8f1-b5744b58b5d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:55.167624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce59b685-3d14-4497-b8f1-b5744b58b5d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:55.168014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce59b685-3d14-4497-b8f1-b5744b58b5d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:55.168741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce59b685-3d14-4497-b8f1-b5744b58b5d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:55.169199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce59b685-3d14-4497-b8f1-b5744b58b5d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:55.169633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce59b685-3d14-4497-b8f1-b5744b58b5d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:55.201177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ce59b685-3d14-4497-b8f1-b5744b58b5d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045769 Aug 24 10:24:55.201730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 451/1039] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:55 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:55.215745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3206b9ae-cf0a-4a20-9080-a0c68fb7fd53 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:55.217890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3206b9ae-cf0a-4a20-9080-a0c68fb7fd53 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:55.224409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3206b9ae-cf0a-4a20-9080-a0c68fb7fd53 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:55.224893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3206b9ae-cf0a-4a20-9080-a0c68fb7fd53 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:55.225334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3206b9ae-cf0a-4a20-9080-a0c68fb7fd53 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:55.226024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3206b9ae-cf0a-4a20-9080-a0c68fb7fd53 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:55.226481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3206b9ae-cf0a-4a20-9080-a0c68fb7fd53 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:55.226906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3206b9ae-cf0a-4a20-9080-a0c68fb7fd53 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:55.252325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:55.257540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3206b9ae-cf0a-4a20-9080-a0c68fb7fd53 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045751 Aug 24 10:24:55.258090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 589/1040] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:55 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:55.306940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:55.307561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:55.313958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:55.314443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:55.314879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:55.315568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:55.316130 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:55.316569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:55.376895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:55.384949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-80c2cc5c-2301-460a-b72d-1af4c35d37ec tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081893 Aug 24 10:24:55.385517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 452/1041] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:55 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:56.213981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b3bdeeb9-147c-444c-a732-00645bebba96 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:56.216626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b3bdeeb9-147c-444c-a732-00645bebba96 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:56.223122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3bdeeb9-147c-444c-a732-00645bebba96 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:56.223600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3bdeeb9-147c-444c-a732-00645bebba96 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:56.224038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3bdeeb9-147c-444c-a732-00645bebba96 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:56.224744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3bdeeb9-147c-444c-a732-00645bebba96 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:56.225196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3bdeeb9-147c-444c-a732-00645bebba96 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:56.225620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3bdeeb9-147c-444c-a732-00645bebba96 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:56.254802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b3bdeeb9-147c-444c-a732-00645bebba96 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044855 Aug 24 10:24:56.255348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 590/1042] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:56 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:56.269944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3fcc338d-2ea5-42a0-b932-aee075f85536 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:56.272084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3fcc338d-2ea5-42a0-b932-aee075f85536 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:56.279117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fcc338d-2ea5-42a0-b932-aee075f85536 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:56.279656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fcc338d-2ea5-42a0-b932-aee075f85536 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:56.280169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fcc338d-2ea5-42a0-b932-aee075f85536 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:56.280883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fcc338d-2ea5-42a0-b932-aee075f85536 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:56.281334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fcc338d-2ea5-42a0-b932-aee075f85536 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:56.281756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3fcc338d-2ea5-42a0-b932-aee075f85536 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:56.309788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3fcc338d-2ea5-42a0-b932-aee075f85536 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043835 Aug 24 10:24:56.310330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 453/1043] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:56 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:56.406741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:56.407359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:56.413351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:56.413838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:56.414279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:56.414971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:56.415532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:56.415963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:56.476439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:56.484125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-756d61cb-90cb-4139-b9b1-45fddb66ce38 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082003 Aug 24 10:24:56.484684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 591/1044] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:56 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:57.267071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c53f5c5c-bacb-4feb-83d6-036f42978899 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:57.269254 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c53f5c5c-bacb-4feb-83d6-036f42978899 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:57.275656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c53f5c5c-bacb-4feb-83d6-036f42978899 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:57.276249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c53f5c5c-bacb-4feb-83d6-036f42978899 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:57.276590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c53f5c5c-bacb-4feb-83d6-036f42978899 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:57.277278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c53f5c5c-bacb-4feb-83d6-036f42978899 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:57.277735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c53f5c5c-bacb-4feb-83d6-036f42978899 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:57.278160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c53f5c5c-bacb-4feb-83d6-036f42978899 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:57.308218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c53f5c5c-bacb-4feb-83d6-036f42978899 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045103 Aug 24 10:24:57.308766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 454/1045] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:57 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:57.321808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b1a5374c-edf8-44cf-8816-5fbafcc35af7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:57.323937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b1a5374c-edf8-44cf-8816-5fbafcc35af7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:57.330361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1a5374c-edf8-44cf-8816-5fbafcc35af7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:57.330847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1a5374c-edf8-44cf-8816-5fbafcc35af7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:57.331286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1a5374c-edf8-44cf-8816-5fbafcc35af7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:57.332272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1a5374c-edf8-44cf-8816-5fbafcc35af7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:57.332721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1a5374c-edf8-44cf-8816-5fbafcc35af7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:57.333146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1a5374c-edf8-44cf-8816-5fbafcc35af7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:57.361879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b1a5374c-edf8-44cf-8816-5fbafcc35af7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044047 Aug 24 10:24:57.362415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 592/1046] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:57 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:57.505992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:57.506623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:57.512576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:57.513108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:57.513537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:57.514229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:57.514692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:57.515118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:57.578001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:57.586080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dd8f4a19-80f2-4f86-ab5d-213ebbda5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084902 Aug 24 10:24:57.586643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 455/1047] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:57 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:58.321151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6e383cfd-3d57-43bf-b130-b3a466c8f7a7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:58.323268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6e383cfd-3d57-43bf-b130-b3a466c8f7a7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:58.329432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e383cfd-3d57-43bf-b130-b3a466c8f7a7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:58.329913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e383cfd-3d57-43bf-b130-b3a466c8f7a7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:58.330350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e383cfd-3d57-43bf-b130-b3a466c8f7a7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:58.331034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e383cfd-3d57-43bf-b130-b3a466c8f7a7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:58.331733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e383cfd-3d57-43bf-b130-b3a466c8f7a7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:58.332191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e383cfd-3d57-43bf-b130-b3a466c8f7a7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:58.361927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6e383cfd-3d57-43bf-b130-b3a466c8f7a7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044677 Aug 24 10:24:58.362469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 593/1048] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:58 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:58.373877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d49cbb2f-b8bb-4339-a2cb-4c2e01c9d7ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:58.376060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d49cbb2f-b8bb-4339-a2cb-4c2e01c9d7ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:58.382574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d49cbb2f-b8bb-4339-a2cb-4c2e01c9d7ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:58.383057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d49cbb2f-b8bb-4339-a2cb-4c2e01c9d7ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:58.383496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d49cbb2f-b8bb-4339-a2cb-4c2e01c9d7ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:58.384221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d49cbb2f-b8bb-4339-a2cb-4c2e01c9d7ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:58.384683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d49cbb2f-b8bb-4339-a2cb-4c2e01c9d7ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:58.385118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d49cbb2f-b8bb-4339-a2cb-4c2e01c9d7ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:58.414605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d49cbb2f-b8bb-4339-a2cb-4c2e01c9d7ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044629 Aug 24 10:24:58.415147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 456/1049] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:58 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:58.607128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:58.607660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:58.613480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:58.613957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:58.614396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:58.615856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:58.615856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:58.616113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:58.675777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:58.683298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b73a2d80-6777-40ff-a47a-71b1b8cbe8cf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080086 Aug 24 10:24:58.683847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 594/1050] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:58 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:24:59.375093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c25e168e-9792-402c-ad56-58fd8f94bdbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:59.377283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c25e168e-9792-402c-ad56-58fd8f94bdbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:59.383742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25e168e-9792-402c-ad56-58fd8f94bdbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:59.384264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25e168e-9792-402c-ad56-58fd8f94bdbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:59.384700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25e168e-9792-402c-ad56-58fd8f94bdbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:59.385389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25e168e-9792-402c-ad56-58fd8f94bdbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:59.385846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25e168e-9792-402c-ad56-58fd8f94bdbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:59.386275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25e168e-9792-402c-ad56-58fd8f94bdbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:59.415006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c25e168e-9792-402c-ad56-58fd8f94bdbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044480 Aug 24 10:24:59.415484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 457/1051] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:59 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:59.426875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d58e3ee1-8397-4083-809a-405084c4e784 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:59.429057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d58e3ee1-8397-4083-809a-405084c4e784 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:59.436015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d58e3ee1-8397-4083-809a-405084c4e784 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:59.436531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d58e3ee1-8397-4083-809a-405084c4e784 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:59.436975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d58e3ee1-8397-4083-809a-405084c4e784 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:59.437671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d58e3ee1-8397-4083-809a-405084c4e784 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:59.438308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d58e3ee1-8397-4083-809a-405084c4e784 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:59.438550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d58e3ee1-8397-4083-809a-405084c4e784 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:59.469219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d58e3ee1-8397-4083-809a-405084c4e784 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046224 Aug 24 10:24:59.469775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 595/1052] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:24:59 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:24:59.703697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:24:59.704337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:24:59.710435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:59.710903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:59.711328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:59.711998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:24:59.712479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:24:59.712896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:24:59.773129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:24:59.781247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-23bc28d1-89e4-4eb7-8e95-7b872b8fa349 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081550 Aug 24 10:24:59.781822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 458/1053] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:24:59 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:00.427308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-58ab609e-32dd-4b57-81b1-20fcb2d7b69a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:00.429481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-58ab609e-32dd-4b57-81b1-20fcb2d7b69a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:00.435986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58ab609e-32dd-4b57-81b1-20fcb2d7b69a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:00.436566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58ab609e-32dd-4b57-81b1-20fcb2d7b69a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:00.437007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58ab609e-32dd-4b57-81b1-20fcb2d7b69a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:00.437699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58ab609e-32dd-4b57-81b1-20fcb2d7b69a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:00.438156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58ab609e-32dd-4b57-81b1-20fcb2d7b69a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:00.438609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58ab609e-32dd-4b57-81b1-20fcb2d7b69a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:00.467668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-58ab609e-32dd-4b57-81b1-20fcb2d7b69a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044777 Aug 24 10:25:00.468238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 596/1054] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:00 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:00.481450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a4ea970a-7dfd-4fb6-82a6-6ca62155a3ab tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:00.483647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a4ea970a-7dfd-4fb6-82a6-6ca62155a3ab tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:00.490050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4ea970a-7dfd-4fb6-82a6-6ca62155a3ab tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:00.490538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4ea970a-7dfd-4fb6-82a6-6ca62155a3ab tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:00.490977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4ea970a-7dfd-4fb6-82a6-6ca62155a3ab tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:00.491667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4ea970a-7dfd-4fb6-82a6-6ca62155a3ab tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:00.492155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4ea970a-7dfd-4fb6-82a6-6ca62155a3ab tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:00.492584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4ea970a-7dfd-4fb6-82a6-6ca62155a3ab tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:00.522217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a4ea970a-7dfd-4fb6-82a6-6ca62155a3ab tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044826 Aug 24 10:25:00.522771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 459/1055] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:00 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:00.801505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:00.802114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:00.807906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:00.808415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:00.808852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:00.810159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:00.810612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:00.811036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:00.873234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:00.881131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9869cd4b-8ff6-407e-9c90-75abdb0b4f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083469 Aug 24 10:25:00.881688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 597/1056] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:00 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:01.479243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a091ae12-99b9-4a0b-a8b3-a39fa684574d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:01.481582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a091ae12-99b9-4a0b-a8b3-a39fa684574d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:01.488287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a091ae12-99b9-4a0b-a8b3-a39fa684574d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:01.488668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a091ae12-99b9-4a0b-a8b3-a39fa684574d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:01.489108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a091ae12-99b9-4a0b-a8b3-a39fa684574d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:01.489812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a091ae12-99b9-4a0b-a8b3-a39fa684574d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:01.490271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a091ae12-99b9-4a0b-a8b3-a39fa684574d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:01.490707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a091ae12-99b9-4a0b-a8b3-a39fa684574d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:01.523786 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a091ae12-99b9-4a0b-a8b3-a39fa684574d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048581 Aug 24 10:25:01.524389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 460/1057] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:01 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:01.534936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b7340d6c-eb54-41bd-a5e6-2b6e0b07b2e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:01.537645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b7340d6c-eb54-41bd-a5e6-2b6e0b07b2e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:01.544612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7340d6c-eb54-41bd-a5e6-2b6e0b07b2e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:01.545102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7340d6c-eb54-41bd-a5e6-2b6e0b07b2e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:01.545542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7340d6c-eb54-41bd-a5e6-2b6e0b07b2e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:01.546242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7340d6c-eb54-41bd-a5e6-2b6e0b07b2e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:01.546703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7340d6c-eb54-41bd-a5e6-2b6e0b07b2e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:01.547133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7340d6c-eb54-41bd-a5e6-2b6e0b07b2e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:01.579081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b7340d6c-eb54-41bd-a5e6-2b6e0b07b2e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048429 Aug 24 10:25:01.579693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 598/1058] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:01 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:01.901164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:01.901708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:01.907694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:01.908225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:01.908667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:01.909387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:01.909845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:01.910272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:01.927701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:01.971750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:01.979405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-32945f40-98ac-4567-85d8-23e81c2db6ba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082202 Aug 24 10:25:01.979960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 461/1059] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:01 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:02.536101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b6cb6f92-5a6e-4e67-be6e-45d51d339ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:02.538222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b6cb6f92-5a6e-4e67-be6e-45d51d339ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:02.544573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6cb6f92-5a6e-4e67-be6e-45d51d339ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:02.545050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6cb6f92-5a6e-4e67-be6e-45d51d339ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:02.545486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6cb6f92-5a6e-4e67-be6e-45d51d339ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:02.546166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6cb6f92-5a6e-4e67-be6e-45d51d339ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:02.546614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6cb6f92-5a6e-4e67-be6e-45d51d339ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:02.547036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6cb6f92-5a6e-4e67-be6e-45d51d339ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:02.576390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b6cb6f92-5a6e-4e67-be6e-45d51d339ab0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044211 Aug 24 10:25:02.576934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 599/1060] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:02 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:02.591492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8bd63a66-35c4-4cfa-944e-a9564e4c0a9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:02.593688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8bd63a66-35c4-4cfa-944e-a9564e4c0a9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:02.600000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd63a66-35c4-4cfa-944e-a9564e4c0a9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:02.600514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd63a66-35c4-4cfa-944e-a9564e4c0a9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:02.600948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd63a66-35c4-4cfa-944e-a9564e4c0a9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:02.601635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd63a66-35c4-4cfa-944e-a9564e4c0a9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:02.602090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd63a66-35c4-4cfa-944e-a9564e4c0a9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:02.602513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd63a66-35c4-4cfa-944e-a9564e4c0a9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:02.631366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8bd63a66-35c4-4cfa-944e-a9564e4c0a9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044519 Aug 24 10:25:02.631911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 462/1061] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:02 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:03.000317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:03.000930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:03.006846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:03.007329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:03.007767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:03.008490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:03.008949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:03.009376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:03.070194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:03.077889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-82b34f94-c74c-449c-a9cf-2a43567fe978 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082005 Aug 24 10:25:03.078452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 600/1062] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:02 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:03.588694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3c8633f7-3b3a-4f17-bb9b-b43b0fd475f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:03.591318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3c8633f7-3b3a-4f17-bb9b-b43b0fd475f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:03.597693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c8633f7-3b3a-4f17-bb9b-b43b0fd475f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:03.598266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c8633f7-3b3a-4f17-bb9b-b43b0fd475f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:03.598661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c8633f7-3b3a-4f17-bb9b-b43b0fd475f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:03.599343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c8633f7-3b3a-4f17-bb9b-b43b0fd475f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:03.599792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c8633f7-3b3a-4f17-bb9b-b43b0fd475f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:03.600235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c8633f7-3b3a-4f17-bb9b-b43b0fd475f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:03.629239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3c8633f7-3b3a-4f17-bb9b-b43b0fd475f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044554 Aug 24 10:25:03.629774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 463/1063] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:03 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:03.644864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-191b54f6-5d50-47d8-9b6f-74a3621aba51 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:03.647012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-191b54f6-5d50-47d8-9b6f-74a3621aba51 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:03.653835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-191b54f6-5d50-47d8-9b6f-74a3621aba51 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:03.654338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-191b54f6-5d50-47d8-9b6f-74a3621aba51 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:03.654765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-191b54f6-5d50-47d8-9b6f-74a3621aba51 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:03.655458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-191b54f6-5d50-47d8-9b6f-74a3621aba51 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:03.655941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-191b54f6-5d50-47d8-9b6f-74a3621aba51 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:03.656400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-191b54f6-5d50-47d8-9b6f-74a3621aba51 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:03.686690 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-191b54f6-5d50-47d8-9b6f-74a3621aba51 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045779 Aug 24 10:25:03.687233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 601/1064] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:03 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:04.097916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:04.098535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:04.104513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:04.104999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:04.105435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:04.106196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:04.106839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:04.107275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:04.167380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:04.175341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b3c19194-947d-47f5-bba2-ded5b92f83b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081321 Aug 24 10:25:04.175906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 464/1065] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:04.641295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3edc1831-d9f4-4a2c-932a-687d6d65340c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:04.643440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3edc1831-d9f4-4a2c-932a-687d6d65340c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:04.649579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3edc1831-d9f4-4a2c-932a-687d6d65340c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:04.650196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3edc1831-d9f4-4a2c-932a-687d6d65340c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:04.650499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3edc1831-d9f4-4a2c-932a-687d6d65340c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:04.651197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3edc1831-d9f4-4a2c-932a-687d6d65340c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:04.651654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3edc1831-d9f4-4a2c-932a-687d6d65340c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:04.652100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3edc1831-d9f4-4a2c-932a-687d6d65340c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:04.681625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3edc1831-d9f4-4a2c-932a-687d6d65340c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044338 Aug 24 10:25:04.682163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 602/1066] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:04 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:04.698809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-485ae336-160c-4aaa-9fc2-d2dbb92fa7d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:04.700955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-485ae336-160c-4aaa-9fc2-d2dbb92fa7d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:04.707521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-485ae336-160c-4aaa-9fc2-d2dbb92fa7d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:04.707994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-485ae336-160c-4aaa-9fc2-d2dbb92fa7d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:04.708446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-485ae336-160c-4aaa-9fc2-d2dbb92fa7d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:04.709121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-485ae336-160c-4aaa-9fc2-d2dbb92fa7d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:04.709568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-485ae336-160c-4aaa-9fc2-d2dbb92fa7d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:04.709985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-485ae336-160c-4aaa-9fc2-d2dbb92fa7d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:04.738973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-485ae336-160c-4aaa-9fc2-d2dbb92fa7d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044141 Aug 24 10:25:04.739510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 465/1067] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:04 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:05.196411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:05.197055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:05.203015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:05.203503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:05.203943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:05.204698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:05.206513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:05.206949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:05.313611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:05.321691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dec2d146-504d-4f70-9175-ecbab36ec9eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.129171 Aug 24 10:25:05.322249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 603/1068] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:05.693942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f22c7f76-f8ce-4677-8860-dc0a1f47e827 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:05.696145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f22c7f76-f8ce-4677-8860-dc0a1f47e827 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:05.702799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22c7f76-f8ce-4677-8860-dc0a1f47e827 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:05.703374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22c7f76-f8ce-4677-8860-dc0a1f47e827 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:05.703704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22c7f76-f8ce-4677-8860-dc0a1f47e827 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:05.704424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22c7f76-f8ce-4677-8860-dc0a1f47e827 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:05.704887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22c7f76-f8ce-4677-8860-dc0a1f47e827 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:05.705310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22c7f76-f8ce-4677-8860-dc0a1f47e827 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:05.735058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f22c7f76-f8ce-4677-8860-dc0a1f47e827 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045141 Aug 24 10:25:05.735602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 466/1069] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:05 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:05.751177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6b0561cf-42ec-40ff-8c73-f540137ccce8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:05.753338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6b0561cf-42ec-40ff-8c73-f540137ccce8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:05.759164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b0561cf-42ec-40ff-8c73-f540137ccce8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:05.759653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b0561cf-42ec-40ff-8c73-f540137ccce8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:05.760122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b0561cf-42ec-40ff-8c73-f540137ccce8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:05.760818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b0561cf-42ec-40ff-8c73-f540137ccce8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:05.761279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b0561cf-42ec-40ff-8c73-f540137ccce8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:05.761712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b0561cf-42ec-40ff-8c73-f540137ccce8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:05.791503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6b0561cf-42ec-40ff-8c73-f540137ccce8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044662 Aug 24 10:25:05.792074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 604/1070] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:05 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:06.342826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:06.343487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:06.349341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:06.349882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:06.350311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:06.350988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:06.351436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:06.351852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:06.464570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:06.472849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-89e104cb-7c38-45cf-9d6d-2603552a5f7e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.133935 Aug 24 10:25:06.473398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 467/1071] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:06 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:06.746883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0c8ee1ea-01fa-408b-8ebf-3d373ad8e805 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:06.749091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0c8ee1ea-01fa-408b-8ebf-3d373ad8e805 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:06.755539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c8ee1ea-01fa-408b-8ebf-3d373ad8e805 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:06.756139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c8ee1ea-01fa-408b-8ebf-3d373ad8e805 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:06.756489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c8ee1ea-01fa-408b-8ebf-3d373ad8e805 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:06.757188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c8ee1ea-01fa-408b-8ebf-3d373ad8e805 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:06.757650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c8ee1ea-01fa-408b-8ebf-3d373ad8e805 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:06.758083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c8ee1ea-01fa-408b-8ebf-3d373ad8e805 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:06.788119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0c8ee1ea-01fa-408b-8ebf-3d373ad8e805 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045115 Aug 24 10:25:06.788871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 605/1072] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:06 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:06.802737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dd2caa5d-681c-43d7-867c-b18ce81d7bc1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:06.805037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dd2caa5d-681c-43d7-867c-b18ce81d7bc1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:06.811387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2caa5d-681c-43d7-867c-b18ce81d7bc1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:06.811867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2caa5d-681c-43d7-867c-b18ce81d7bc1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:06.812341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2caa5d-681c-43d7-867c-b18ce81d7bc1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:06.813026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2caa5d-681c-43d7-867c-b18ce81d7bc1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:06.813483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2caa5d-681c-43d7-867c-b18ce81d7bc1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:06.813906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2caa5d-681c-43d7-867c-b18ce81d7bc1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:06.843981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dd2caa5d-681c-43d7-867c-b18ce81d7bc1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045210 Aug 24 10:25:06.844556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 468/1073] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:06 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:07.493756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:07.494320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:07.500883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:07.501369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:07.501809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:07.502506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:07.502993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:07.503422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:07.521597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:07.566053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:07.573786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3e27d6e9-0f3e-47c8-8f52-11f225f550c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083971 Aug 24 10:25:07.574342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 606/1074] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:07 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:07.800651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3a7f24d5-e3dd-4c2a-84e5-527b0ef315ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:07.802860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3a7f24d5-e3dd-4c2a-84e5-527b0ef315ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:07.810010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a7f24d5-e3dd-4c2a-84e5-527b0ef315ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:07.810489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a7f24d5-e3dd-4c2a-84e5-527b0ef315ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:07.810922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a7f24d5-e3dd-4c2a-84e5-527b0ef315ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:07.811615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a7f24d5-e3dd-4c2a-84e5-527b0ef315ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:07.812084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a7f24d5-e3dd-4c2a-84e5-527b0ef315ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:07.812510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a7f24d5-e3dd-4c2a-84e5-527b0ef315ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:07.841512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3a7f24d5-e3dd-4c2a-84e5-527b0ef315ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044837 Aug 24 10:25:07.842076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 469/1075] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:07 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:07.856819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-180cebeb-4ec5-473d-b7a9-54e829decfbd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:07.859004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-180cebeb-4ec5-473d-b7a9-54e829decfbd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:07.865877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-180cebeb-4ec5-473d-b7a9-54e829decfbd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:07.866369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-180cebeb-4ec5-473d-b7a9-54e829decfbd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:07.866811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-180cebeb-4ec5-473d-b7a9-54e829decfbd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:07.867518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-180cebeb-4ec5-473d-b7a9-54e829decfbd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:07.867988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-180cebeb-4ec5-473d-b7a9-54e829decfbd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:07.868450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-180cebeb-4ec5-473d-b7a9-54e829decfbd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:07.898540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-180cebeb-4ec5-473d-b7a9-54e829decfbd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046299 Aug 24 10:25:07.899089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 607/1076] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:07 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:08.595577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:08.596251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:08.602545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:08.603030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:08.603471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:08.604191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:08.604652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:08.605083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:08.665445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:08.673196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9f3e2663-3ff4-48e1-b673-96996e1660e4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081527 Aug 24 10:25:08.673756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 470/1077] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:08 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:08.853776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3a282306-487d-4934-a890-c0860e0ac0de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:08.856189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3a282306-487d-4934-a890-c0860e0ac0de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:08.862501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a282306-487d-4934-a890-c0860e0ac0de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:08.863094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a282306-487d-4934-a890-c0860e0ac0de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:08.863403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a282306-487d-4934-a890-c0860e0ac0de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:08.864105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a282306-487d-4934-a890-c0860e0ac0de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:08.864558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a282306-487d-4934-a890-c0860e0ac0de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:08.864978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a282306-487d-4934-a890-c0860e0ac0de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:08.896718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3a282306-487d-4934-a890-c0860e0ac0de tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047006 Aug 24 10:25:08.897260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 608/1078] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:08.910926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1a038826-1039-4496-a169-c7207c058ac8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:08.913196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1a038826-1039-4496-a169-c7207c058ac8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:08.920938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a038826-1039-4496-a169-c7207c058ac8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:08.925468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a038826-1039-4496-a169-c7207c058ac8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:08.925468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a038826-1039-4496-a169-c7207c058ac8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:08.925468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a038826-1039-4496-a169-c7207c058ac8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:08.925468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a038826-1039-4496-a169-c7207c058ac8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:08.925858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a038826-1039-4496-a169-c7207c058ac8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:08.953157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1a038826-1039-4496-a169-c7207c058ac8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046369 Aug 24 10:25:08.953704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 471/1079] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:08 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:09.694216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:09.694853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:09.701361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:09.701841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:09.702271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:09.702951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:09.703401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:09.703823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:09.766606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:09.774347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-092d5134-0e58-4465-846f-d3cf0cbfb079 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083976 Aug 24 10:25:09.774895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 609/1080] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:09 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:09.909169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-50398392-c2f0-4ecf-afa8-2974dfbeb795 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:09.911347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-50398392-c2f0-4ecf-afa8-2974dfbeb795 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:09.917931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50398392-c2f0-4ecf-afa8-2974dfbeb795 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:09.919569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50398392-c2f0-4ecf-afa8-2974dfbeb795 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:09.919569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50398392-c2f0-4ecf-afa8-2974dfbeb795 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:09.920023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50398392-c2f0-4ecf-afa8-2974dfbeb795 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:09.920503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50398392-c2f0-4ecf-afa8-2974dfbeb795 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:09.920928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50398392-c2f0-4ecf-afa8-2974dfbeb795 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:09.950415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-50398392-c2f0-4ecf-afa8-2974dfbeb795 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045424 Aug 24 10:25:09.950883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 472/1081] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:09 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:09.964747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2be6e320-9b5b-45f4-adb9-5f36ae2aac43 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:09.966969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2be6e320-9b5b-45f4-adb9-5f36ae2aac43 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:09.973501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2be6e320-9b5b-45f4-adb9-5f36ae2aac43 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:09.973983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2be6e320-9b5b-45f4-adb9-5f36ae2aac43 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:09.974425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2be6e320-9b5b-45f4-adb9-5f36ae2aac43 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:09.975125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2be6e320-9b5b-45f4-adb9-5f36ae2aac43 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:09.975589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2be6e320-9b5b-45f4-adb9-5f36ae2aac43 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:09.976020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2be6e320-9b5b-45f4-adb9-5f36ae2aac43 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:10.006218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2be6e320-9b5b-45f4-adb9-5f36ae2aac43 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045351 Aug 24 10:25:10.006774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 610/1082] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:09 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:10.794990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:10.795619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:10.802540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:10.803048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:10.803488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:10.804259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:10.804717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:10.805149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:10.868427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:10.876828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-360bdcc3-195f-4b46-8715-9793247c3449 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085833 Aug 24 10:25:10.878629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 473/1083] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:10 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:10.963254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e5ad1279-3c2f-41ac-9ef2-4eaeacd1fbe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:10.965503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e5ad1279-3c2f-41ac-9ef2-4eaeacd1fbe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:10.972143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5ad1279-3c2f-41ac-9ef2-4eaeacd1fbe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:10.972662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5ad1279-3c2f-41ac-9ef2-4eaeacd1fbe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:10.973104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5ad1279-3c2f-41ac-9ef2-4eaeacd1fbe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:10.973810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5ad1279-3c2f-41ac-9ef2-4eaeacd1fbe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:10.974267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5ad1279-3c2f-41ac-9ef2-4eaeacd1fbe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:10.974694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e5ad1279-3c2f-41ac-9ef2-4eaeacd1fbe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:11.005246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e5ad1279-3c2f-41ac-9ef2-4eaeacd1fbe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045928 Aug 24 10:25:11.005814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 611/1084] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:10 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:11.019054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3340f860-0674-4c0b-903b-dacd436162be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:11.021588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3340f860-0674-4c0b-903b-dacd436162be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:11.028029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3340f860-0674-4c0b-903b-dacd436162be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:11.028537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3340f860-0674-4c0b-903b-dacd436162be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:11.028971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3340f860-0674-4c0b-903b-dacd436162be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:11.029663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3340f860-0674-4c0b-903b-dacd436162be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:11.030113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3340f860-0674-4c0b-903b-dacd436162be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:11.030532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3340f860-0674-4c0b-903b-dacd436162be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:11.061097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3340f860-0674-4c0b-903b-dacd436162be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045963 Aug 24 10:25:11.061662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 474/1085] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:11 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:11.899511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:11.900172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:11.916903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:11.917518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:11.918059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:11.918931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:11.920105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:11.920619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:11.985411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:11.993286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2751323e-9705-441f-934f-a3022bfac972 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.097697 Aug 24 10:25:11.993830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 612/1086] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:11 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:12.018023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6d577518-7ccf-4759-8d82-2ed51757e425 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:12.020180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6d577518-7ccf-4759-8d82-2ed51757e425 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:12.027214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d577518-7ccf-4759-8d82-2ed51757e425 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:12.027632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d577518-7ccf-4759-8d82-2ed51757e425 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:12.028155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d577518-7ccf-4759-8d82-2ed51757e425 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:12.029382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d577518-7ccf-4759-8d82-2ed51757e425 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:12.029915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d577518-7ccf-4759-8d82-2ed51757e425 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:12.030265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d577518-7ccf-4759-8d82-2ed51757e425 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:12.058967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6d577518-7ccf-4759-8d82-2ed51757e425 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045000 Aug 24 10:25:12.059576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 475/1087] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:12.073837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f5e74147-a4e3-4c83-a9bf-f2b428b4ae56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:12.076637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f5e74147-a4e3-4c83-a9bf-f2b428b4ae56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:12.084737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e74147-a4e3-4c83-a9bf-f2b428b4ae56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:12.085237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e74147-a4e3-4c83-a9bf-f2b428b4ae56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:12.085774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e74147-a4e3-4c83-a9bf-f2b428b4ae56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:12.086810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e74147-a4e3-4c83-a9bf-f2b428b4ae56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:12.087607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e74147-a4e3-4c83-a9bf-f2b428b4ae56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:12.088107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e74147-a4e3-4c83-a9bf-f2b428b4ae56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:12.119306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f5e74147-a4e3-4c83-a9bf-f2b428b4ae56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050030 Aug 24 10:25:12.119864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 613/1088] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:12 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:13.014092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:13.014704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:13.021417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:13.021999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:13.022258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:13.022960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:13.023422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:13.023854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:13.071427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-017bda51-6318-4d99-8f0d-098db7ff132e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:13.073579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-017bda51-6318-4d99-8f0d-098db7ff132e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:13.082766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-017bda51-6318-4d99-8f0d-098db7ff132e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:13.083298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-017bda51-6318-4d99-8f0d-098db7ff132e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:13.083732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-017bda51-6318-4d99-8f0d-098db7ff132e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:13.084443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-017bda51-6318-4d99-8f0d-098db7ff132e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:13.085019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-017bda51-6318-4d99-8f0d-098db7ff132e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:13.085454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-017bda51-6318-4d99-8f0d-098db7ff132e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:13.089034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:13.097674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-af7d8751-a2c8-4aa1-89fb-14fc674f1631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087954 Aug 24 10:25:13.098585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 476/1089] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:13 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:13.117270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-017bda51-6318-4d99-8f0d-098db7ff132e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050249 Aug 24 10:25:13.117941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 614/1090] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:13 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:13.130675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2d6704ba-3de3-4bb3-9e2f-8fb6ba0a3f01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:13.132836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2d6704ba-3de3-4bb3-9e2f-8fb6ba0a3f01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:13.139613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d6704ba-3de3-4bb3-9e2f-8fb6ba0a3f01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:13.140151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d6704ba-3de3-4bb3-9e2f-8fb6ba0a3f01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:13.140585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d6704ba-3de3-4bb3-9e2f-8fb6ba0a3f01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:13.141269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d6704ba-3de3-4bb3-9e2f-8fb6ba0a3f01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:13.141719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d6704ba-3de3-4bb3-9e2f-8fb6ba0a3f01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:13.142136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d6704ba-3de3-4bb3-9e2f-8fb6ba0a3f01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:13.174151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2d6704ba-3de3-4bb3-9e2f-8fb6ba0a3f01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047333 Aug 24 10:25:13.174758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 477/1091] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:13 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:14.118318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:14.118868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:14.126599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:14.127168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:14.127696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:14.128484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:14.128766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7b731154-b769-4fbb-8d2d-4dc80d9125ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:14.129193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:14.129468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:14.131016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7b731154-b769-4fbb-8d2d-4dc80d9125ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:14.137941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b731154-b769-4fbb-8d2d-4dc80d9125ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:14.138427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b731154-b769-4fbb-8d2d-4dc80d9125ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:14.138859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b731154-b769-4fbb-8d2d-4dc80d9125ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:14.139573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b731154-b769-4fbb-8d2d-4dc80d9125ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:14.140018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b731154-b769-4fbb-8d2d-4dc80d9125ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:14.140464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b731154-b769-4fbb-8d2d-4dc80d9125ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:14.144185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:14.169670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7b731154-b769-4fbb-8d2d-4dc80d9125ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044861 Aug 24 10:25:14.170211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 478/1092] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:14 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:14.186919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-82d13162-80f3-412c-934d-c74a025c0bb2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:14.189082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-82d13162-80f3-412c-934d-c74a025c0bb2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:14.195550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82d13162-80f3-412c-934d-c74a025c0bb2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:14.196059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82d13162-80f3-412c-934d-c74a025c0bb2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:14.196353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:14.196501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82d13162-80f3-412c-934d-c74a025c0bb2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:14.197195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82d13162-80f3-412c-934d-c74a025c0bb2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:14.197646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82d13162-80f3-412c-934d-c74a025c0bb2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:14.198068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82d13162-80f3-412c-934d-c74a025c0bb2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:14.204841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-54f0ed27-9fc6-48eb-a3e4-78f97e665d1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.091000 Aug 24 10:25:14.205406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 615/1093] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:14 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:14.228345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-82d13162-80f3-412c-934d-c74a025c0bb2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045779 Aug 24 10:25:14.228893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 479/1094] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:14 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:15.181918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1bd35aae-7f8d-4781-b7ca-7c778495751d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:15.184092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1bd35aae-7f8d-4781-b7ca-7c778495751d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:15.190962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bd35aae-7f8d-4781-b7ca-7c778495751d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:15.191447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bd35aae-7f8d-4781-b7ca-7c778495751d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:15.191888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bd35aae-7f8d-4781-b7ca-7c778495751d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:15.192616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bd35aae-7f8d-4781-b7ca-7c778495751d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:15.193073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bd35aae-7f8d-4781-b7ca-7c778495751d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:15.193500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bd35aae-7f8d-4781-b7ca-7c778495751d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:15.223220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1bd35aae-7f8d-4781-b7ca-7c778495751d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045182 Aug 24 10:25:15.223776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 616/1095] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:15 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:15.225585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:15.226248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:15.232924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:15.233398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:15.233829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:15.234540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:15.234985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:15.235403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:15.240551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c32dafa2-faf1-45be-97f7-57ffd4cae92f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:15.242664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c32dafa2-faf1-45be-97f7-57ffd4cae92f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:15.248542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c32dafa2-faf1-45be-97f7-57ffd4cae92f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:15.249031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c32dafa2-faf1-45be-97f7-57ffd4cae92f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:15.249471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c32dafa2-faf1-45be-97f7-57ffd4cae92f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:15.250162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c32dafa2-faf1-45be-97f7-57ffd4cae92f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:15.250619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c32dafa2-faf1-45be-97f7-57ffd4cae92f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:15.251046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c32dafa2-faf1-45be-97f7-57ffd4cae92f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:15.282553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c32dafa2-faf1-45be-97f7-57ffd4cae92f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045752 Aug 24 10:25:15.283121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 617/1096] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:15 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:15.300761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:15.308474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5275f453-d811-48a3-8429-21cb2a76bc66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086729 Aug 24 10:25:15.309018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 480/1097] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:15 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:25:16.236036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f76acb05-9f83-4aa6-a247-d01cb5f91633 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:16.238181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f76acb05-9f83-4aa6-a247-d01cb5f91633 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:16.244616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f76acb05-9f83-4aa6-a247-d01cb5f91633 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:16.245101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f76acb05-9f83-4aa6-a247-d01cb5f91633 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:16.245543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f76acb05-9f83-4aa6-a247-d01cb5f91633 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:16.246244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f76acb05-9f83-4aa6-a247-d01cb5f91633 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:16.246706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f76acb05-9f83-4aa6-a247-d01cb5f91633 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:16.247140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f76acb05-9f83-4aa6-a247-d01cb5f91633 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:16.275911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f76acb05-9f83-4aa6-a247-d01cb5f91633 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044462 Aug 24 10:25:16.276486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 618/1098] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:16 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:16.294153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f18d0655-46c9-4493-a06d-43520b51c5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:16.296923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f18d0655-46c9-4493-a06d-43520b51c5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:16.303167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f18d0655-46c9-4493-a06d-43520b51c5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:16.303641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f18d0655-46c9-4493-a06d-43520b51c5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:16.304094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f18d0655-46c9-4493-a06d-43520b51c5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:16.304776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f18d0655-46c9-4493-a06d-43520b51c5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:16.305233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f18d0655-46c9-4493-a06d-43520b51c5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:16.305672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f18d0655-46c9-4493-a06d-43520b51c5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:16.333099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:16.333218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:16.334835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f18d0655-46c9-4493-a06d-43520b51c5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044536 Aug 24 10:25:16.335499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 481/1099] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:16 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:16.340783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:16.341322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:16.341843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:16.343042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:16.343490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:16.343910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:16.404130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:16.412223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-778d3ba1-8d13-4482-8c89-cf31d816b8e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087379 Aug 24 10:25:16.412769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 619/1100] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:16 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:25:17.288846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-074a4f0b-46da-42aa-b448-927802ade8b3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:17.291072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-074a4f0b-46da-42aa-b448-927802ade8b3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:17.297500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-074a4f0b-46da-42aa-b448-927802ade8b3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:17.298058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-074a4f0b-46da-42aa-b448-927802ade8b3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:17.298414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-074a4f0b-46da-42aa-b448-927802ade8b3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:17.299108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-074a4f0b-46da-42aa-b448-927802ade8b3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:17.299566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-074a4f0b-46da-42aa-b448-927802ade8b3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:17.299997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-074a4f0b-46da-42aa-b448-927802ade8b3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:17.330661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-074a4f0b-46da-42aa-b448-927802ade8b3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045826 Aug 24 10:25:17.331471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 482/1101] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:17 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:17.347866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-49a23640-8e5f-4c2f-bb20-f17a998d21b1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:17.350025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-49a23640-8e5f-4c2f-bb20-f17a998d21b1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:17.356314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49a23640-8e5f-4c2f-bb20-f17a998d21b1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:17.356794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49a23640-8e5f-4c2f-bb20-f17a998d21b1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:17.357225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49a23640-8e5f-4c2f-bb20-f17a998d21b1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:17.358443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49a23640-8e5f-4c2f-bb20-f17a998d21b1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:17.358897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49a23640-8e5f-4c2f-bb20-f17a998d21b1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:17.359315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49a23640-8e5f-4c2f-bb20-f17a998d21b1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:17.388095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-49a23640-8e5f-4c2f-bb20-f17a998d21b1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044063 Aug 24 10:25:17.388624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 620/1102] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:17 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:17.433592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:17.434209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:17.440773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:17.441248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:17.441677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:17.442350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:17.442796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:17.443212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:17.503499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:17.511288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f57b0a13-ffd1-44f8-8b91-d61076068e9a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081601 Aug 24 10:25:17.511859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 483/1103] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:17 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:18.344560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-20efd1cd-3e19-49cc-ac46-51af51abce31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:18.346717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-20efd1cd-3e19-49cc-ac46-51af51abce31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:18.352743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20efd1cd-3e19-49cc-ac46-51af51abce31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:18.353392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20efd1cd-3e19-49cc-ac46-51af51abce31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:18.353651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20efd1cd-3e19-49cc-ac46-51af51abce31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:18.354343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20efd1cd-3e19-49cc-ac46-51af51abce31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:18.354796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20efd1cd-3e19-49cc-ac46-51af51abce31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:18.355226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20efd1cd-3e19-49cc-ac46-51af51abce31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:18.385416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-20efd1cd-3e19-49cc-ac46-51af51abce31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045416 Aug 24 10:25:18.385966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 621/1104] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:18 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:18.400482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e904f283-433d-4d29-9bd0-14214e719c55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:18.402613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e904f283-433d-4d29-9bd0-14214e719c55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:18.404899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=86 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:18.411305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e904f283-433d-4d29-9bd0-14214e719c55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:18.411782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e904f283-433d-4d29-9bd0-14214e719c55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:18.412241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e904f283-433d-4d29-9bd0-14214e719c55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:18.412926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e904f283-433d-4d29-9bd0-14214e719c55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:18.413372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e904f283-433d-4d29-9bd0-14214e719c55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:18.413793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e904f283-433d-4d29-9bd0-14214e719c55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:18.444191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e904f283-433d-4d29-9bd0-14214e719c55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047513 Aug 24 10:25:18.444841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 484/1105] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:18 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:18.532065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:18.532683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:18.538614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:18.539099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:18.539535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:18.540258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:18.540719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:18.541149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:18.605359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:18.613661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e60ac6b9-72ca-42c9-863a-4c131f715876 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086027 Aug 24 10:25:18.614207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 622/1106] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:18 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:19.397991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cbda714c-71d3-4a6c-8b51-2a61c247aec2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:19.400220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cbda714c-71d3-4a6c-8b51-2a61c247aec2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:19.406259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbda714c-71d3-4a6c-8b51-2a61c247aec2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:19.406835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbda714c-71d3-4a6c-8b51-2a61c247aec2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:19.407173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbda714c-71d3-4a6c-8b51-2a61c247aec2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:19.407860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbda714c-71d3-4a6c-8b51-2a61c247aec2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:19.408348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbda714c-71d3-4a6c-8b51-2a61c247aec2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:19.408773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbda714c-71d3-4a6c-8b51-2a61c247aec2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:19.438762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cbda714c-71d3-4a6c-8b51-2a61c247aec2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044956 Aug 24 10:25:19.439317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 485/1107] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:19 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:19.456535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f4784d6a-e304-4a8b-a4c9-a969999bca93 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:19.458779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f4784d6a-e304-4a8b-a4c9-a969999bca93 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:19.461090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=86 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:19.466826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4784d6a-e304-4a8b-a4c9-a969999bca93 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:19.467309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4784d6a-e304-4a8b-a4c9-a969999bca93 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:19.467744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4784d6a-e304-4a8b-a4c9-a969999bca93 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:19.468488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4784d6a-e304-4a8b-a4c9-a969999bca93 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:19.468937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4784d6a-e304-4a8b-a4c9-a969999bca93 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:19.469357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4784d6a-e304-4a8b-a4c9-a969999bca93 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:19.472896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:19.498626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f4784d6a-e304-4a8b-a4c9-a969999bca93 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046257 Aug 24 10:25:19.499165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 623/1108] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:19 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:19.636623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:19.637243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:19.643196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:19.643681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:19.644143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:19.644842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:19.645297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:19.645724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:19.706805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:19.714589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8153c4bb-4e52-4a7c-8980-0b5440b9cc2b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081984 Aug 24 10:25:19.715148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 486/1109] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:19 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:20.451128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7cbde631-2744-4711-bcef-248cf2492689 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:20.453779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7cbde631-2744-4711-bcef-248cf2492689 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:20.460273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7cbde631-2744-4711-bcef-248cf2492689 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:20.460847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7cbde631-2744-4711-bcef-248cf2492689 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:20.461200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7cbde631-2744-4711-bcef-248cf2492689 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:20.461903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7cbde631-2744-4711-bcef-248cf2492689 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:20.462362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7cbde631-2744-4711-bcef-248cf2492689 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:20.462800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7cbde631-2744-4711-bcef-248cf2492689 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:20.494478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7cbde631-2744-4711-bcef-248cf2492689 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047452 Aug 24 10:25:20.495027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 624/1110] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:20 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:20.510683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1953594b-2930-4dd9-8618-bd131e11d6d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:20.512887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1953594b-2930-4dd9-8618-bd131e11d6d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:20.519809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1953594b-2930-4dd9-8618-bd131e11d6d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:20.520327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1953594b-2930-4dd9-8618-bd131e11d6d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:20.520760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1953594b-2930-4dd9-8618-bd131e11d6d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:20.521521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1953594b-2930-4dd9-8618-bd131e11d6d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:20.522008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1953594b-2930-4dd9-8618-bd131e11d6d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:20.522433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1953594b-2930-4dd9-8618-bd131e11d6d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:20.551533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1953594b-2930-4dd9-8618-bd131e11d6d8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044754 Aug 24 10:25:20.552103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 487/1111] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:20 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:20.735102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:20.735634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:20.741497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:20.741979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:20.742418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:20.743105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:20.743569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:20.743999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:20.806659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:20.814792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-42cee83f-1a30-4144-ac7f-2ffa07763f29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083676 Aug 24 10:25:20.815334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 625/1112] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:20 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:21.506736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-87861917-16b2-4da1-a2fe-bd163796b0a1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:21.508936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-87861917-16b2-4da1-a2fe-bd163796b0a1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:21.514872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87861917-16b2-4da1-a2fe-bd163796b0a1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:21.515440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87861917-16b2-4da1-a2fe-bd163796b0a1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:21.515758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87861917-16b2-4da1-a2fe-bd163796b0a1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:21.516462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87861917-16b2-4da1-a2fe-bd163796b0a1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:21.516910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87861917-16b2-4da1-a2fe-bd163796b0a1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:21.517334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87861917-16b2-4da1-a2fe-bd163796b0a1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:21.547643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-87861917-16b2-4da1-a2fe-bd163796b0a1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044850 Aug 24 10:25:21.548211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 488/1113] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:21 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:21.564109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eec22a2f-b1b5-4793-881a-2169a6d05759 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:21.566305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eec22a2f-b1b5-4793-881a-2169a6d05759 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:21.572728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eec22a2f-b1b5-4793-881a-2169a6d05759 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:21.573215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eec22a2f-b1b5-4793-881a-2169a6d05759 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:21.573659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eec22a2f-b1b5-4793-881a-2169a6d05759 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:21.574357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eec22a2f-b1b5-4793-881a-2169a6d05759 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:21.574818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eec22a2f-b1b5-4793-881a-2169a6d05759 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:21.575248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eec22a2f-b1b5-4793-881a-2169a6d05759 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:21.604740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eec22a2f-b1b5-4793-881a-2169a6d05759 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044557 Aug 24 10:25:21.605463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 626/1114] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:21 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:21.836154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:21.836888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:21.842752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:21.844088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:21.844554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:21.845231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:21.845679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:21.846091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:21.906763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:21.915476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-62988b0b-52f9-4190-a390-b57907159272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083189 Aug 24 10:25:21.916108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 489/1115] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:21 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:22.558862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d4119a14-36a5-4ee3-a706-92a695aad2a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:22.561067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d4119a14-36a5-4ee3-a706-92a695aad2a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:22.567533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4119a14-36a5-4ee3-a706-92a695aad2a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:22.568228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4119a14-36a5-4ee3-a706-92a695aad2a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:22.568461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4119a14-36a5-4ee3-a706-92a695aad2a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:22.569141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4119a14-36a5-4ee3-a706-92a695aad2a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:22.569589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4119a14-36a5-4ee3-a706-92a695aad2a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:22.570010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4119a14-36a5-4ee3-a706-92a695aad2a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:22.599120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d4119a14-36a5-4ee3-a706-92a695aad2a9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044191 Aug 24 10:25:22.599669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 627/1116] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:22 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:22.616898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9f3d8347-bd62-42e9-beba-bca2b3d134f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:22.619047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9f3d8347-bd62-42e9-beba-bca2b3d134f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:22.626163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3d8347-bd62-42e9-beba-bca2b3d134f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:22.626655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3d8347-bd62-42e9-beba-bca2b3d134f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:22.627093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3d8347-bd62-42e9-beba-bca2b3d134f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:22.627785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3d8347-bd62-42e9-beba-bca2b3d134f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:22.628275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3d8347-bd62-42e9-beba-bca2b3d134f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:22.628705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f3d8347-bd62-42e9-beba-bca2b3d134f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:22.657223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9f3d8347-bd62-42e9-beba-bca2b3d134f4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044262 Aug 24 10:25:22.657768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 490/1117] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:22 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:22.938946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:22.939563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:22.945626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:22.946126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:22.946602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:22.947299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:22.947760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:22.948252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:23.009920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:23.017627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0cf90e77-d334-4a33-ab40-08b3fc94c631 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082740 Aug 24 10:25:23.018178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 628/1118] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:22 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:23.611510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4fdcdf76-e53c-4049-9f80-8359b8e5ba5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:23.613678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4fdcdf76-e53c-4049-9f80-8359b8e5ba5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:23.619656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fdcdf76-e53c-4049-9f80-8359b8e5ba5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:23.620259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fdcdf76-e53c-4049-9f80-8359b8e5ba5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:23.620594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fdcdf76-e53c-4049-9f80-8359b8e5ba5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:23.621282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fdcdf76-e53c-4049-9f80-8359b8e5ba5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:23.621738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fdcdf76-e53c-4049-9f80-8359b8e5ba5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:23.622170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fdcdf76-e53c-4049-9f80-8359b8e5ba5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:23.652224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4fdcdf76-e53c-4049-9f80-8359b8e5ba5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044774 Aug 24 10:25:23.652769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 491/1119] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:23 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:23.669714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3f23b0c5-a38c-4e47-8670-c7bc85b98e15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:23.671865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3f23b0c5-a38c-4e47-8670-c7bc85b98e15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:23.678145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f23b0c5-a38c-4e47-8670-c7bc85b98e15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:23.678645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f23b0c5-a38c-4e47-8670-c7bc85b98e15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:23.679057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f23b0c5-a38c-4e47-8670-c7bc85b98e15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:23.679735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f23b0c5-a38c-4e47-8670-c7bc85b98e15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:23.680222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f23b0c5-a38c-4e47-8670-c7bc85b98e15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:23.680646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f23b0c5-a38c-4e47-8670-c7bc85b98e15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:23.710854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3f23b0c5-a38c-4e47-8670-c7bc85b98e15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045077 Aug 24 10:25:23.711408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 629/1120] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:23 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:24.038764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:24.039504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:24.045748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:24.046716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:24.047154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:24.047843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:24.048324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:24.048752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:24.111741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:24.119563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-15788f6d-444f-46ae-8979-a69b40315ed7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084674 Aug 24 10:25:24.120143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 492/1121] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:24 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:24.667943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-466ab2a7-a5d4-4fa8-bf69-7a2511d1bcd0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:24.670137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-466ab2a7-a5d4-4fa8-bf69-7a2511d1bcd0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:24.677283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-466ab2a7-a5d4-4fa8-bf69-7a2511d1bcd0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:24.677771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-466ab2a7-a5d4-4fa8-bf69-7a2511d1bcd0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:24.678210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-466ab2a7-a5d4-4fa8-bf69-7a2511d1bcd0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:24.678909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-466ab2a7-a5d4-4fa8-bf69-7a2511d1bcd0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:24.679359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-466ab2a7-a5d4-4fa8-bf69-7a2511d1bcd0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:24.679781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-466ab2a7-a5d4-4fa8-bf69-7a2511d1bcd0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:24.709070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-466ab2a7-a5d4-4fa8-bf69-7a2511d1bcd0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047375 Aug 24 10:25:24.709615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 630/1122] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:24 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:24.722719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f97d3537-e4e1-47b9-b2fc-a145e1eed481 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:24.724968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f97d3537-e4e1-47b9-b2fc-a145e1eed481 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:24.731344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f97d3537-e4e1-47b9-b2fc-a145e1eed481 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:24.731821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f97d3537-e4e1-47b9-b2fc-a145e1eed481 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:24.732293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f97d3537-e4e1-47b9-b2fc-a145e1eed481 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:24.732985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f97d3537-e4e1-47b9-b2fc-a145e1eed481 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:24.733954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f97d3537-e4e1-47b9-b2fc-a145e1eed481 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:24.734378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f97d3537-e4e1-47b9-b2fc-a145e1eed481 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:24.763504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f97d3537-e4e1-47b9-b2fc-a145e1eed481 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044680 Aug 24 10:25:24.764263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 493/1123] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:24 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:25.140894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:25.141505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:25.147560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:25.148070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:25.148507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:25.149195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:25.149650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:25.150075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:25.214800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:25.222988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f8b7121e-26be-4419-b33f-08bd06cfe07c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086010 Aug 24 10:25:25.223479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 631/1124] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:25 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:25.720788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4ec0a780-86b9-48cc-b08c-605323361baa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:25.722937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4ec0a780-86b9-48cc-b08c-605323361baa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:25.729439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ec0a780-86b9-48cc-b08c-605323361baa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:25.729926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ec0a780-86b9-48cc-b08c-605323361baa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:25.730367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ec0a780-86b9-48cc-b08c-605323361baa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:25.731064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ec0a780-86b9-48cc-b08c-605323361baa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:25.731523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ec0a780-86b9-48cc-b08c-605323361baa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:25.731952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ec0a780-86b9-48cc-b08c-605323361baa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:25.757396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:25.762670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4ec0a780-86b9-48cc-b08c-605323361baa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046021 Aug 24 10:25:25.763234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 494/1125] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:25 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:25.775843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-398b47d2-2f66-4da9-be43-ddea44db8ffb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:25.778166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-398b47d2-2f66-4da9-be43-ddea44db8ffb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:25.785544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-398b47d2-2f66-4da9-be43-ddea44db8ffb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:25.786033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-398b47d2-2f66-4da9-be43-ddea44db8ffb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:25.786540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-398b47d2-2f66-4da9-be43-ddea44db8ffb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:25.787281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-398b47d2-2f66-4da9-be43-ddea44db8ffb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:25.787769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-398b47d2-2f66-4da9-be43-ddea44db8ffb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:25.788310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-398b47d2-2f66-4da9-be43-ddea44db8ffb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:25.820353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-398b47d2-2f66-4da9-be43-ddea44db8ffb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048787 Aug 24 10:25:25.820967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 632/1126] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:25 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:26.244488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:26.245092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:26.251192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:26.251672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:26.252157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:26.253421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:26.253866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:26.254281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:26.319503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:26.328029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b5164abc-6d0a-4f49-afe4-3f3dfea2b059 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087783 Aug 24 10:25:26.328617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 495/1127] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:26 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:26.776960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c675f5e3-681d-4350-bfc9-02033a74f21a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:26.779464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c675f5e3-681d-4350-bfc9-02033a74f21a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:26.787438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c675f5e3-681d-4350-bfc9-02033a74f21a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:26.787936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c675f5e3-681d-4350-bfc9-02033a74f21a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:26.788411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c675f5e3-681d-4350-bfc9-02033a74f21a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:26.789178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c675f5e3-681d-4350-bfc9-02033a74f21a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:26.789590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c675f5e3-681d-4350-bfc9-02033a74f21a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:26.790014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c675f5e3-681d-4350-bfc9-02033a74f21a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:26.820575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c675f5e3-681d-4350-bfc9-02033a74f21a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048722 Aug 24 10:25:26.821129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 633/1128] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:26 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:26.833506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e93ca88c-4ee8-49fb-b0a2-262c16d7f8d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:26.835646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e93ca88c-4ee8-49fb-b0a2-262c16d7f8d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:26.842033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e93ca88c-4ee8-49fb-b0a2-262c16d7f8d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:26.842511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e93ca88c-4ee8-49fb-b0a2-262c16d7f8d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:26.842940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e93ca88c-4ee8-49fb-b0a2-262c16d7f8d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:26.843624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e93ca88c-4ee8-49fb-b0a2-262c16d7f8d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:26.844093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e93ca88c-4ee8-49fb-b0a2-262c16d7f8d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:26.844555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e93ca88c-4ee8-49fb-b0a2-262c16d7f8d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:26.874340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e93ca88c-4ee8-49fb-b0a2-262c16d7f8d1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045078 Aug 24 10:25:26.874892 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 496/1129] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:26 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:27.350678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:27.351289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:27.357190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:27.357673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:27.358110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:27.358790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:27.359242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:27.359668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:27.427060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:27.434759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e0821509-b4b1-44f1-ac20-97dd0a8e6fad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.088006 Aug 24 10:25:27.435305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 634/1130] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:27 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:27.832994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-42854188-a6d8-4880-a082-028132d1555e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:27.835343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-42854188-a6d8-4880-a082-028132d1555e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:27.841438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42854188-a6d8-4880-a082-028132d1555e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:27.841921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42854188-a6d8-4880-a082-028132d1555e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:27.842360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42854188-a6d8-4880-a082-028132d1555e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:27.843055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42854188-a6d8-4880-a082-028132d1555e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:27.843515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42854188-a6d8-4880-a082-028132d1555e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:27.843938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42854188-a6d8-4880-a082-028132d1555e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:27.872887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-42854188-a6d8-4880-a082-028132d1555e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044814 Aug 24 10:25:27.873439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 497/1131] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:27 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:27.886636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c419b2ee-f4ca-4c5f-bb2b-0a3ca0167e01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:27.888782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c419b2ee-f4ca-4c5f-bb2b-0a3ca0167e01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:27.895040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419b2ee-f4ca-4c5f-bb2b-0a3ca0167e01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:27.895517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419b2ee-f4ca-4c5f-bb2b-0a3ca0167e01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:27.895948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419b2ee-f4ca-4c5f-bb2b-0a3ca0167e01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:27.896656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419b2ee-f4ca-4c5f-bb2b-0a3ca0167e01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:27.897106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419b2ee-f4ca-4c5f-bb2b-0a3ca0167e01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:27.897524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419b2ee-f4ca-4c5f-bb2b-0a3ca0167e01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:27.925992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c419b2ee-f4ca-4c5f-bb2b-0a3ca0167e01 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043236 Aug 24 10:25:27.927130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 635/1132] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:27 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:28.454762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:28.455655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:28.462321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:28.462818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:28.463234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:28.463916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:28.464394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:28.464814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:28.524837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:28.532598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0fc1f4ab-0a46-489e-a833-a819ffc73fa0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081707 Aug 24 10:25:28.533153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 498/1133] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:28 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:28.885395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c01ef460-bdcf-4e56-910a-31cf3be2e90e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:28.887586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c01ef460-bdcf-4e56-910a-31cf3be2e90e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:28.894041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c01ef460-bdcf-4e56-910a-31cf3be2e90e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:28.894518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c01ef460-bdcf-4e56-910a-31cf3be2e90e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:28.894954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c01ef460-bdcf-4e56-910a-31cf3be2e90e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:28.896286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c01ef460-bdcf-4e56-910a-31cf3be2e90e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:28.896746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c01ef460-bdcf-4e56-910a-31cf3be2e90e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:28.897170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c01ef460-bdcf-4e56-910a-31cf3be2e90e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:28.925799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c01ef460-bdcf-4e56-910a-31cf3be2e90e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044355 Aug 24 10:25:28.926341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 636/1134] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:28 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:28.938766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d932bc08-7e8a-4e76-b1c8-04bcf8e712ed tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:28.941012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d932bc08-7e8a-4e76-b1c8-04bcf8e712ed tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:28.947891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d932bc08-7e8a-4e76-b1c8-04bcf8e712ed tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:28.948434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d932bc08-7e8a-4e76-b1c8-04bcf8e712ed tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:28.948871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d932bc08-7e8a-4e76-b1c8-04bcf8e712ed tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:28.949565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d932bc08-7e8a-4e76-b1c8-04bcf8e712ed tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:28.950024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d932bc08-7e8a-4e76-b1c8-04bcf8e712ed tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:28.950452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d932bc08-7e8a-4e76-b1c8-04bcf8e712ed tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:28.980393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d932bc08-7e8a-4e76-b1c8-04bcf8e712ed tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045576 Aug 24 10:25:28.980941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 499/1135] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:28 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:29.553920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:29.554539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:29.560524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:29.561004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:29.561447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:29.562138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:29.562594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:29.563022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:29.624470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:29.632629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-85d65244-ce01-4001-8b0e-6aa645501d05 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082589 Aug 24 10:25:29.633183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 637/1136] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:29 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:29.938345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-15f5701a-dc9a-45c0-a0ee-5cbeabb765d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:29.940545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-15f5701a-dc9a-45c0-a0ee-5cbeabb765d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:29.947387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15f5701a-dc9a-45c0-a0ee-5cbeabb765d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:29.947866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15f5701a-dc9a-45c0-a0ee-5cbeabb765d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:29.948330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15f5701a-dc9a-45c0-a0ee-5cbeabb765d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:29.949016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15f5701a-dc9a-45c0-a0ee-5cbeabb765d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:29.949469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15f5701a-dc9a-45c0-a0ee-5cbeabb765d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:29.949893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15f5701a-dc9a-45c0-a0ee-5cbeabb765d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:29.978993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-15f5701a-dc9a-45c0-a0ee-5cbeabb765d2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044613 Aug 24 10:25:29.979545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 500/1137] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:29 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:29.993216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dc89b508-54a0-4943-bace-bef1170c2de7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:29.995339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dc89b508-54a0-4943-bace-bef1170c2de7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:30.001828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc89b508-54a0-4943-bace-bef1170c2de7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:30.002294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc89b508-54a0-4943-bace-bef1170c2de7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:30.002739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc89b508-54a0-4943-bace-bef1170c2de7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:30.003426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc89b508-54a0-4943-bace-bef1170c2de7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:30.003884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc89b508-54a0-4943-bace-bef1170c2de7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:30.004408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc89b508-54a0-4943-bace-bef1170c2de7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:30.033724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dc89b508-54a0-4943-bace-bef1170c2de7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045007 Aug 24 10:25:30.034270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 638/1138] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:29 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:30.654763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:30.655470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:30.661968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:30.662534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:30.663010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:30.664425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:30.664986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:30.665531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:30.728557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:30.736268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e1ee8416-503f-4171-b915-b4c3b213b386 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085723 Aug 24 10:25:30.736825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 501/1139] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:30 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:30.991385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-322c0a53-4130-43d4-9626-c93337d7bf98 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:30.994189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-322c0a53-4130-43d4-9626-c93337d7bf98 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:31.000662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-322c0a53-4130-43d4-9626-c93337d7bf98 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:31.001182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-322c0a53-4130-43d4-9626-c93337d7bf98 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:31.001599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-322c0a53-4130-43d4-9626-c93337d7bf98 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:31.002278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-322c0a53-4130-43d4-9626-c93337d7bf98 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:31.002727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-322c0a53-4130-43d4-9626-c93337d7bf98 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:31.003146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-322c0a53-4130-43d4-9626-c93337d7bf98 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:31.028256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:31.033573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-322c0a53-4130-43d4-9626-c93337d7bf98 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046148 Aug 24 10:25:31.034125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 639/1140] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:30 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:31.045706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d6996938-e148-43c5-bfab-e32fdfd8c696 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:31.047868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d6996938-e148-43c5-bfab-e32fdfd8c696 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:31.055068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6996938-e148-43c5-bfab-e32fdfd8c696 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:31.055556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6996938-e148-43c5-bfab-e32fdfd8c696 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:31.055990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6996938-e148-43c5-bfab-e32fdfd8c696 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:31.056721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6996938-e148-43c5-bfab-e32fdfd8c696 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:31.057178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6996938-e148-43c5-bfab-e32fdfd8c696 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:31.057607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6996938-e148-43c5-bfab-e32fdfd8c696 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:31.086234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d6996938-e148-43c5-bfab-e32fdfd8c696 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044513 Aug 24 10:25:31.086787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 502/1141] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:31 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:31.756381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:31.756988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:31.763460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:31.763954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:31.764426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:31.765136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:31.765596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:31.766027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:31.826537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:31.834759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-72dde5cc-a01a-45e8-bb2f-05d329746027 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082556 Aug 24 10:25:31.835931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 640/1142] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:31 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:32.046983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-abc68dad-61ce-41ee-bb0f-bfc95c36688c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:32.049103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-abc68dad-61ce-41ee-bb0f-bfc95c36688c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:32.056153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abc68dad-61ce-41ee-bb0f-bfc95c36688c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:32.056638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abc68dad-61ce-41ee-bb0f-bfc95c36688c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:32.057075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abc68dad-61ce-41ee-bb0f-bfc95c36688c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:32.057769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abc68dad-61ce-41ee-bb0f-bfc95c36688c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:32.058226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abc68dad-61ce-41ee-bb0f-bfc95c36688c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:32.058659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abc68dad-61ce-41ee-bb0f-bfc95c36688c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:32.087931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-abc68dad-61ce-41ee-bb0f-bfc95c36688c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045051 Aug 24 10:25:32.088518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 503/1143] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:32 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:32.098339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b8505d66-7872-4cdc-a4e5-373d117e4735 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:32.100541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b8505d66-7872-4cdc-a4e5-373d117e4735 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:32.106925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8505d66-7872-4cdc-a4e5-373d117e4735 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:32.107415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8505d66-7872-4cdc-a4e5-373d117e4735 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:32.107861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8505d66-7872-4cdc-a4e5-373d117e4735 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:32.108604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8505d66-7872-4cdc-a4e5-373d117e4735 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:32.109069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8505d66-7872-4cdc-a4e5-373d117e4735 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:32.109501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8505d66-7872-4cdc-a4e5-373d117e4735 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:32.139100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b8505d66-7872-4cdc-a4e5-373d117e4735 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044696 Aug 24 10:25:32.139642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 641/1144] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:32 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:32.855782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:32.856446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:32.863118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:32.863645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:32.864208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:32.865034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:32.865633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:32.866130 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:32.932301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:32.941328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4d91273f-9734-4c7a-8fcd-998e408069cc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.089393 Aug 24 10:25:32.942002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 504/1145] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:32 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:33.100645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5af91d0f-b0c0-4a49-89c9-b25834dea1a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:33.102737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5af91d0f-b0c0-4a49-89c9-b25834dea1a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:33.109790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af91d0f-b0c0-4a49-89c9-b25834dea1a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:33.110279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af91d0f-b0c0-4a49-89c9-b25834dea1a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:33.110718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af91d0f-b0c0-4a49-89c9-b25834dea1a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:33.111429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af91d0f-b0c0-4a49-89c9-b25834dea1a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:33.111889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af91d0f-b0c0-4a49-89c9-b25834dea1a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:33.112351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5af91d0f-b0c0-4a49-89c9-b25834dea1a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:33.141196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5af91d0f-b0c0-4a49-89c9-b25834dea1a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044677 Aug 24 10:25:33.141743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 642/1146] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:33 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:33.150811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6253c7d8-a3df-4d25-83fc-912b2f4a2f0b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:33.153032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6253c7d8-a3df-4d25-83fc-912b2f4a2f0b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:33.159388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6253c7d8-a3df-4d25-83fc-912b2f4a2f0b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:33.159872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6253c7d8-a3df-4d25-83fc-912b2f4a2f0b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:33.160343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6253c7d8-a3df-4d25-83fc-912b2f4a2f0b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:33.161034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6253c7d8-a3df-4d25-83fc-912b2f4a2f0b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:33.161499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6253c7d8-a3df-4d25-83fc-912b2f4a2f0b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:33.161922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6253c7d8-a3df-4d25-83fc-912b2f4a2f0b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:33.191441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6253c7d8-a3df-4d25-83fc-912b2f4a2f0b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044533 Aug 24 10:25:33.191989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 505/1147] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:33 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:33.963299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:33.964124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:33.971946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:33.972652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:33.973143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:33.973925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:33.974450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:33.974932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:34.038451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:34.046240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3da8f98d-5fcd-4a65-b2bc-0c0d22d596ea tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087456 Aug 24 10:25:34.046913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 643/1148] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:33 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:34.154167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-599f5982-84c5-45d4-a360-081da0d59d2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:34.156631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-599f5982-84c5-45d4-a360-081da0d59d2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:34.163237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-599f5982-84c5-45d4-a360-081da0d59d2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:34.163719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-599f5982-84c5-45d4-a360-081da0d59d2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:34.164183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-599f5982-84c5-45d4-a360-081da0d59d2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:34.164868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-599f5982-84c5-45d4-a360-081da0d59d2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:34.165324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-599f5982-84c5-45d4-a360-081da0d59d2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:34.165747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-599f5982-84c5-45d4-a360-081da0d59d2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:34.194656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-599f5982-84c5-45d4-a360-081da0d59d2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044891 Aug 24 10:25:34.195228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 506/1149] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:34 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:34.203506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-95c252d7-89d2-4ab9-9b0d-72d34d1cf5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:34.205966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-95c252d7-89d2-4ab9-9b0d-72d34d1cf5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:34.212373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95c252d7-89d2-4ab9-9b0d-72d34d1cf5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:34.212863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95c252d7-89d2-4ab9-9b0d-72d34d1cf5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:34.213305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95c252d7-89d2-4ab9-9b0d-72d34d1cf5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:34.214001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95c252d7-89d2-4ab9-9b0d-72d34d1cf5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:34.214466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95c252d7-89d2-4ab9-9b0d-72d34d1cf5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:34.214897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95c252d7-89d2-4ab9-9b0d-72d34d1cf5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:34.244542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-95c252d7-89d2-4ab9-9b0d-72d34d1cf5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045577 Aug 24 10:25:34.245083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 644/1150] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:34 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:35.068567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:35.069208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:35.075240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:35.075736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:35.076200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:35.076894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:35.077918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:35.078346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:35.142569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:35.150320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2edbebe7-c98f-4c1f-aad4-ee0eec9f3b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085623 Aug 24 10:25:35.150881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 507/1151] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:35 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:35.207219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c81522c9-1534-4b55-ad76-5f9fa7d4cd2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:35.209417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c81522c9-1534-4b55-ad76-5f9fa7d4cd2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:35.215696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c81522c9-1534-4b55-ad76-5f9fa7d4cd2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:35.216218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c81522c9-1534-4b55-ad76-5f9fa7d4cd2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:35.216662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c81522c9-1534-4b55-ad76-5f9fa7d4cd2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:35.217352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c81522c9-1534-4b55-ad76-5f9fa7d4cd2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:35.217813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c81522c9-1534-4b55-ad76-5f9fa7d4cd2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:35.218251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c81522c9-1534-4b55-ad76-5f9fa7d4cd2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:35.248096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c81522c9-1534-4b55-ad76-5f9fa7d4cd2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044937 Aug 24 10:25:35.248575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 645/1152] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:35 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:35.257449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2048b16d-0aa3-41d7-b24d-25b8f074e780 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:35.259630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2048b16d-0aa3-41d7-b24d-25b8f074e780 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:35.266054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2048b16d-0aa3-41d7-b24d-25b8f074e780 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:35.266541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2048b16d-0aa3-41d7-b24d-25b8f074e780 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:35.266984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2048b16d-0aa3-41d7-b24d-25b8f074e780 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:35.267670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2048b16d-0aa3-41d7-b24d-25b8f074e780 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:35.268155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2048b16d-0aa3-41d7-b24d-25b8f074e780 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:35.268585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2048b16d-0aa3-41d7-b24d-25b8f074e780 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:35.298499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2048b16d-0aa3-41d7-b24d-25b8f074e780 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044942 Aug 24 10:25:35.299058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 508/1153] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:35 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:36.171683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:36.172264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:36.178569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:36.179048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:36.179484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:36.180269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:36.180836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:36.181346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:36.247892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:36.257378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bcd559b5-ed28-4c09-abde-240ba847ecc3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.090338 Aug 24 10:25:36.258199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 646/1154] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:36 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:36.261048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-540cad54-3962-45a6-a8ae-3d4397f3c9c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:36.263194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-540cad54-3962-45a6-a8ae-3d4397f3c9c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:36.270218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-540cad54-3962-45a6-a8ae-3d4397f3c9c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:36.270695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-540cad54-3962-45a6-a8ae-3d4397f3c9c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:36.271132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-540cad54-3962-45a6-a8ae-3d4397f3c9c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:36.271826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-540cad54-3962-45a6-a8ae-3d4397f3c9c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:36.272320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-540cad54-3962-45a6-a8ae-3d4397f3c9c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:36.272749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-540cad54-3962-45a6-a8ae-3d4397f3c9c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:36.301358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-540cad54-3962-45a6-a8ae-3d4397f3c9c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044457 Aug 24 10:25:36.301910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 509/1155] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:36 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:36.310625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2d289725-187b-485e-842b-33654e0f6710 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:36.312870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2d289725-187b-485e-842b-33654e0f6710 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:36.319173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d289725-187b-485e-842b-33654e0f6710 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:36.319649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d289725-187b-485e-842b-33654e0f6710 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:36.320109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d289725-187b-485e-842b-33654e0f6710 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:36.320798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d289725-187b-485e-842b-33654e0f6710 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:36.321252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d289725-187b-485e-842b-33654e0f6710 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:36.321674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d289725-187b-485e-842b-33654e0f6710 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:36.354432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2d289725-187b-485e-842b-33654e0f6710 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047722 Aug 24 10:25:36.355021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 647/1156] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:36 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:37.278845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:37.279462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:37.285895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:37.286387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:37.286825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:37.287515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:37.287970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:37.288421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:37.313895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fc58ec68-1161-4809-8ff0-b98ade784388 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:37.316189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fc58ec68-1161-4809-8ff0-b98ade784388 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:37.322539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc58ec68-1161-4809-8ff0-b98ade784388 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:37.323016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc58ec68-1161-4809-8ff0-b98ade784388 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:37.323447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc58ec68-1161-4809-8ff0-b98ade784388 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:37.324161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc58ec68-1161-4809-8ff0-b98ade784388 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:37.324612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc58ec68-1161-4809-8ff0-b98ade784388 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:37.325039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc58ec68-1161-4809-8ff0-b98ade784388 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:37.351882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:37.357348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fc58ec68-1161-4809-8ff0-b98ade784388 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047878 Aug 24 10:25:37.358118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 648/1157] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:37 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:37.361257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f9e57fab-5b45-445e-96bd-280d12bf85f6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086262 Aug 24 10:25:37.361911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 510/1158] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:37 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:37.366913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d9704f14-e45f-41eb-9026-7c62792f539e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:37.369053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d9704f14-e45f-41eb-9026-7c62792f539e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:37.374938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9704f14-e45f-41eb-9026-7c62792f539e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:37.375421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9704f14-e45f-41eb-9026-7c62792f539e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:37.376443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9704f14-e45f-41eb-9026-7c62792f539e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:37.377219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9704f14-e45f-41eb-9026-7c62792f539e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:37.377671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9704f14-e45f-41eb-9026-7c62792f539e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:37.378098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9704f14-e45f-41eb-9026-7c62792f539e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:37.405903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d9704f14-e45f-41eb-9026-7c62792f539e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042690 Aug 24 10:25:37.406440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 649/1159] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:37 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:38.370503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7c4732c3-cd02-4d35-83b4-3abd8ce5e212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:38.372691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7c4732c3-cd02-4d35-83b4-3abd8ce5e212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:38.379368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c4732c3-cd02-4d35-83b4-3abd8ce5e212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:38.379858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c4732c3-cd02-4d35-83b4-3abd8ce5e212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:38.380323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c4732c3-cd02-4d35-83b4-3abd8ce5e212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:38.381027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c4732c3-cd02-4d35-83b4-3abd8ce5e212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:38.381488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c4732c3-cd02-4d35-83b4-3abd8ce5e212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:38.381917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c4732c3-cd02-4d35-83b4-3abd8ce5e212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:38.382834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:38.383459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:38.390539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:38.391078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:38.392084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:38.393115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:38.393602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:38.394037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:38.400750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:38.415333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7c4732c3-cd02-4d35-83b4-3abd8ce5e212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048744 Aug 24 10:25:38.416095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 511/1160] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:38 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:38.420702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-36dd43e8-9d0e-4c69-9354-b08c26bfc072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:38.422857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-36dd43e8-9d0e-4c69-9354-b08c26bfc072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:38.428714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36dd43e8-9d0e-4c69-9354-b08c26bfc072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:38.429221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36dd43e8-9d0e-4c69-9354-b08c26bfc072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:38.429646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36dd43e8-9d0e-4c69-9354-b08c26bfc072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:38.430343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36dd43e8-9d0e-4c69-9354-b08c26bfc072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:38.430803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36dd43e8-9d0e-4c69-9354-b08c26bfc072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:38.431231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36dd43e8-9d0e-4c69-9354-b08c26bfc072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:38.463678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-36dd43e8-9d0e-4c69-9354-b08c26bfc072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046536 Aug 24 10:25:38.464517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 512/1161] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:38 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:38.470344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:38.478661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1d9f52d9-3cb5-4f03-9440-c54369dd2f19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.099812 Aug 24 10:25:38.479286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 650/1162] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:38 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:25:39.427758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2c46d5c7-327f-44ac-a6c2-ec8932e25b65 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:39.429995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2c46d5c7-327f-44ac-a6c2-ec8932e25b65 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:39.436446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c46d5c7-327f-44ac-a6c2-ec8932e25b65 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:39.436931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c46d5c7-327f-44ac-a6c2-ec8932e25b65 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:39.437362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c46d5c7-327f-44ac-a6c2-ec8932e25b65 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:39.438055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c46d5c7-327f-44ac-a6c2-ec8932e25b65 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:39.438509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c46d5c7-327f-44ac-a6c2-ec8932e25b65 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:39.438930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c46d5c7-327f-44ac-a6c2-ec8932e25b65 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:39.471553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2c46d5c7-327f-44ac-a6c2-ec8932e25b65 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048113 Aug 24 10:25:39.472222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 513/1163] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:39 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:39.475489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-02439e0f-ece3-4d72-bb5c-c6e4cf294ec5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:39.478035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-02439e0f-ece3-4d72-bb5c-c6e4cf294ec5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:39.484181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02439e0f-ece3-4d72-bb5c-c6e4cf294ec5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:39.484661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02439e0f-ece3-4d72-bb5c-c6e4cf294ec5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:39.485106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02439e0f-ece3-4d72-bb5c-c6e4cf294ec5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:39.485784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02439e0f-ece3-4d72-bb5c-c6e4cf294ec5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:39.486237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02439e0f-ece3-4d72-bb5c-c6e4cf294ec5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:39.486658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02439e0f-ece3-4d72-bb5c-c6e4cf294ec5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:39.501803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:39.502974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:39.509533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:39.510017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:39.510451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:39.511188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:39.511685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:39.512133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:39.515993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-02439e0f-ece3-4d72-bb5c-c6e4cf294ec5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044469 Aug 24 10:25:39.516576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 651/1164] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:39 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:39.574115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:39.581982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ef5ef969-c83d-4756-9682-525e2246836a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085133 Aug 24 10:25:39.582533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 514/1165] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:39 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:25:40.484276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e4e9bd62-23d9-42c0-83ab-86d6cfcf6e21 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:40.486480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e4e9bd62-23d9-42c0-83ab-86d6cfcf6e21 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:40.492821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e9bd62-23d9-42c0-83ab-86d6cfcf6e21 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:40.493303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e9bd62-23d9-42c0-83ab-86d6cfcf6e21 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:40.493734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e9bd62-23d9-42c0-83ab-86d6cfcf6e21 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:40.494420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e9bd62-23d9-42c0-83ab-86d6cfcf6e21 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:40.494869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e9bd62-23d9-42c0-83ab-86d6cfcf6e21 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:40.495289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e9bd62-23d9-42c0-83ab-86d6cfcf6e21 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:40.524405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e4e9bd62-23d9-42c0-83ab-86d6cfcf6e21 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044057 Aug 24 10:25:40.524946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 652/1166] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:40 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:40.528916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d12c92ac-20c6-46b3-bff9-248d84c4c93a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:40.531109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d12c92ac-20c6-46b3-bff9-248d84c4c93a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:40.538290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12c92ac-20c6-46b3-bff9-248d84c4c93a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:40.538775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12c92ac-20c6-46b3-bff9-248d84c4c93a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:40.539212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12c92ac-20c6-46b3-bff9-248d84c4c93a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:40.539908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12c92ac-20c6-46b3-bff9-248d84c4c93a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:40.540398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12c92ac-20c6-46b3-bff9-248d84c4c93a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:40.540824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12c92ac-20c6-46b3-bff9-248d84c4c93a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:40.569804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d12c92ac-20c6-46b3-bff9-248d84c4c93a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044872 Aug 24 10:25:40.570368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 515/1167] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:40 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:40.603135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:40.603748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:40.609704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:40.610220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:40.610658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:40.611344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:40.611799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:40.612266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:40.672076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:40.679793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d5621c2d-3223-44d6-b5c5-f2ebc7fadc74 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080581 Aug 24 10:25:40.680379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 653/1168] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:40 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:41.537916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8b4966a0-e16f-477f-9bd4-3d4b9a7bceda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:41.540141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8b4966a0-e16f-477f-9bd4-3d4b9a7bceda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:41.546517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8b4966a0-e16f-477f-9bd4-3d4b9a7bceda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:41.547076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8b4966a0-e16f-477f-9bd4-3d4b9a7bceda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:41.547513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8b4966a0-e16f-477f-9bd4-3d4b9a7bceda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:41.548251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8b4966a0-e16f-477f-9bd4-3d4b9a7bceda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:41.548716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8b4966a0-e16f-477f-9bd4-3d4b9a7bceda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:41.549706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8b4966a0-e16f-477f-9bd4-3d4b9a7bceda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:41.578893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8b4966a0-e16f-477f-9bd4-3d4b9a7bceda tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046176 Aug 24 10:25:41.579451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 516/1169] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:41 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:41.582499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-28ae8f6b-c285-4675-982a-8f85907b71ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:41.584699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-28ae8f6b-c285-4675-982a-8f85907b71ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:41.591004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28ae8f6b-c285-4675-982a-8f85907b71ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:41.591495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28ae8f6b-c285-4675-982a-8f85907b71ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:41.591938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28ae8f6b-c285-4675-982a-8f85907b71ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:41.592657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28ae8f6b-c285-4675-982a-8f85907b71ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:41.593119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28ae8f6b-c285-4675-982a-8f85907b71ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:41.593549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28ae8f6b-c285-4675-982a-8f85907b71ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:41.623269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-28ae8f6b-c285-4675-982a-8f85907b71ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045020 Aug 24 10:25:41.623814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 654/1170] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:41 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:41.700625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:41.701240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:41.707491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:41.707972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:41.708445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:41.709146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:41.709607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:41.710029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:41.773438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:41.781160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2258e0d0-a765-4061-a770-52aada46c339 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084408 Aug 24 10:25:41.782037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 517/1171] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:41 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:42.591349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-45e87f5c-f59b-4d89-bef8-8e448c4bfc3a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:42.593402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-45e87f5c-f59b-4d89-bef8-8e448c4bfc3a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:42.600381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45e87f5c-f59b-4d89-bef8-8e448c4bfc3a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:42.600848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45e87f5c-f59b-4d89-bef8-8e448c4bfc3a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:42.601284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45e87f5c-f59b-4d89-bef8-8e448c4bfc3a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:42.601966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45e87f5c-f59b-4d89-bef8-8e448c4bfc3a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:42.602415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45e87f5c-f59b-4d89-bef8-8e448c4bfc3a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:42.602837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45e87f5c-f59b-4d89-bef8-8e448c4bfc3a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:42.630891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-45e87f5c-f59b-4d89-bef8-8e448c4bfc3a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043543 Aug 24 10:25:42.631440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 655/1172] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:42 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:42.634812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-10a90848-1024-4710-b2b5-1cc3999fdfd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:42.637140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-10a90848-1024-4710-b2b5-1cc3999fdfd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:42.644303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10a90848-1024-4710-b2b5-1cc3999fdfd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:42.644784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10a90848-1024-4710-b2b5-1cc3999fdfd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:42.645217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10a90848-1024-4710-b2b5-1cc3999fdfd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:42.645910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10a90848-1024-4710-b2b5-1cc3999fdfd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:42.646361 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10a90848-1024-4710-b2b5-1cc3999fdfd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:42.646784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10a90848-1024-4710-b2b5-1cc3999fdfd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:42.676274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-10a90848-1024-4710-b2b5-1cc3999fdfd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045352 Aug 24 10:25:42.676819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 518/1173] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:42 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:42.803835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:42.804633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:42.811291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:42.811774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:42.812244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:42.812937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:42.813502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:42.813810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:42.832960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:42.882565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:42.890172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b0ea8c2d-4af3-43cb-9775-1a8b73675501 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.091686 Aug 24 10:25:42.890718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 656/1174] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:42 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:43.642494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b134326c-cd47-44f6-b17a-4a388f10be89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:43.644698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b134326c-cd47-44f6-b17a-4a388f10be89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:43.651161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b134326c-cd47-44f6-b17a-4a388f10be89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:43.651657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b134326c-cd47-44f6-b17a-4a388f10be89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:43.652119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b134326c-cd47-44f6-b17a-4a388f10be89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:43.652818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b134326c-cd47-44f6-b17a-4a388f10be89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:43.653274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b134326c-cd47-44f6-b17a-4a388f10be89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:43.653699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b134326c-cd47-44f6-b17a-4a388f10be89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:43.683453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b134326c-cd47-44f6-b17a-4a388f10be89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044220 Aug 24 10:25:43.683453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 519/1175] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:43 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:43.688751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f3595f01-a350-4c12-8a61-f568be2b3ef0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:43.691585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f3595f01-a350-4c12-8a61-f568be2b3ef0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:43.698014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3595f01-a350-4c12-8a61-f568be2b3ef0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:43.698503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3595f01-a350-4c12-8a61-f568be2b3ef0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:43.698944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3595f01-a350-4c12-8a61-f568be2b3ef0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:43.699638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3595f01-a350-4c12-8a61-f568be2b3ef0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:43.700194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3595f01-a350-4c12-8a61-f568be2b3ef0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:43.700626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3595f01-a350-4c12-8a61-f568be2b3ef0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:43.729563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f3595f01-a350-4c12-8a61-f568be2b3ef0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044850 Aug 24 10:25:43.730100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 657/1176] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:43 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:43.911397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:43.911932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:43.917849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:43.918337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:43.918773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:43.919458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:43.919919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:43.920369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:43.984289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:43.993726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fa124785-7065-4623-a858-c51a0a788a02 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086799 Aug 24 10:25:43.994290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 520/1177] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:43 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:44.695395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9236ab4d-15a4-4001-8c73-80a385ee193c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:44.697554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9236ab4d-15a4-4001-8c73-80a385ee193c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:44.704029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9236ab4d-15a4-4001-8c73-80a385ee193c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:44.704538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9236ab4d-15a4-4001-8c73-80a385ee193c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:44.705320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9236ab4d-15a4-4001-8c73-80a385ee193c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:44.706520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9236ab4d-15a4-4001-8c73-80a385ee193c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:44.706983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9236ab4d-15a4-4001-8c73-80a385ee193c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:44.707414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9236ab4d-15a4-4001-8c73-80a385ee193c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:44.737153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9236ab4d-15a4-4001-8c73-80a385ee193c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045717 Aug 24 10:25:44.737702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 658/1178] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:44 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:44.741840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1086a196-0447-498d-b155-8523c78bba05 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:44.744022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1086a196-0447-498d-b155-8523c78bba05 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:44.750307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1086a196-0447-498d-b155-8523c78bba05 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:44.750792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1086a196-0447-498d-b155-8523c78bba05 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:44.751221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1086a196-0447-498d-b155-8523c78bba05 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:44.752425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1086a196-0447-498d-b155-8523c78bba05 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:44.752875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1086a196-0447-498d-b155-8523c78bba05 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:44.753292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1086a196-0447-498d-b155-8523c78bba05 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:44.784722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1086a196-0447-498d-b155-8523c78bba05 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046281 Aug 24 10:25:44.784722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 521/1179] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:44 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:45.013806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:45.014320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:45.020249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:45.020729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:45.021163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:45.021841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:45.022290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:45.022712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:45.086597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:45.094238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7c122e38-9db1-4b9d-8bc7-da411f4094b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084377 Aug 24 10:25:45.094813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 659/1180] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:45 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:45.749720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a3ea4cb2-3853-4a32-8351-74b831931ffc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:45.751903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a3ea4cb2-3853-4a32-8351-74b831931ffc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:45.758502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3ea4cb2-3853-4a32-8351-74b831931ffc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:45.758984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3ea4cb2-3853-4a32-8351-74b831931ffc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:45.759484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3ea4cb2-3853-4a32-8351-74b831931ffc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:45.760248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3ea4cb2-3853-4a32-8351-74b831931ffc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:45.760729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3ea4cb2-3853-4a32-8351-74b831931ffc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:45.761156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3ea4cb2-3853-4a32-8351-74b831931ffc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:45.793243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a3ea4cb2-3853-4a32-8351-74b831931ffc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047614 Aug 24 10:25:45.793811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 522/1181] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:45 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:45.796593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2ec18ed5-2d0f-4703-9070-d9405a3e2ac6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:45.798798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2ec18ed5-2d0f-4703-9070-d9405a3e2ac6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:45.805090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ec18ed5-2d0f-4703-9070-d9405a3e2ac6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:45.805567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ec18ed5-2d0f-4703-9070-d9405a3e2ac6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:45.805997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ec18ed5-2d0f-4703-9070-d9405a3e2ac6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:45.806702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ec18ed5-2d0f-4703-9070-d9405a3e2ac6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:45.807152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ec18ed5-2d0f-4703-9070-d9405a3e2ac6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:45.807575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ec18ed5-2d0f-4703-9070-d9405a3e2ac6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:45.837817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2ec18ed5-2d0f-4703-9070-d9405a3e2ac6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045314 Aug 24 10:25:45.838381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 660/1182] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:45 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:46.114991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:46.115510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:46.121389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:46.121861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:46.122287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:46.122958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:46.123403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:46.123821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:46.184429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:46.192559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7d12f22a-5266-4ee6-8f80-149fb3b34cfb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081562 Aug 24 10:25:46.193120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 523/1183] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:46 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:46.806328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1c85cd64-773c-46d1-8804-40a68a44ae7c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:46.808525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1c85cd64-773c-46d1-8804-40a68a44ae7c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:46.814811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c85cd64-773c-46d1-8804-40a68a44ae7c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:46.815291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c85cd64-773c-46d1-8804-40a68a44ae7c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:46.815727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c85cd64-773c-46d1-8804-40a68a44ae7c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:46.816442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c85cd64-773c-46d1-8804-40a68a44ae7c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:46.816892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c85cd64-773c-46d1-8804-40a68a44ae7c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:46.817314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c85cd64-773c-46d1-8804-40a68a44ae7c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:46.845719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1c85cd64-773c-46d1-8804-40a68a44ae7c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043380 Aug 24 10:25:46.846985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 661/1184] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:46 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:46.850776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b2a32247-5fe0-45b4-b9ed-4c9ca7f177e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:46.852978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b2a32247-5fe0-45b4-b9ed-4c9ca7f177e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:46.859488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2a32247-5fe0-45b4-b9ed-4c9ca7f177e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:46.859975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2a32247-5fe0-45b4-b9ed-4c9ca7f177e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:46.860447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2a32247-5fe0-45b4-b9ed-4c9ca7f177e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:46.861143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2a32247-5fe0-45b4-b9ed-4c9ca7f177e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:46.861602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2a32247-5fe0-45b4-b9ed-4c9ca7f177e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:46.863436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2a32247-5fe0-45b4-b9ed-4c9ca7f177e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:46.891626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b2a32247-5fe0-45b4-b9ed-4c9ca7f177e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045477 Aug 24 10:25:46.892187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 524/1185] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:46 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:47.214306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:47.214928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:47.220872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:47.222015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:47.222459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:47.223152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:47.223608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:47.224036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:47.282931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:47.291250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-936d025d-fc47-4041-996d-8975480e8d89 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080846 Aug 24 10:25:47.291812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 662/1186] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:47 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:47.858384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ee570d34-bdd4-4f4c-9c17-e46d6c2915d6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:47.860628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ee570d34-bdd4-4f4c-9c17-e46d6c2915d6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:47.867479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee570d34-bdd4-4f4c-9c17-e46d6c2915d6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:47.867959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee570d34-bdd4-4f4c-9c17-e46d6c2915d6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:47.868428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee570d34-bdd4-4f4c-9c17-e46d6c2915d6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:47.869111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee570d34-bdd4-4f4c-9c17-e46d6c2915d6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:47.869564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee570d34-bdd4-4f4c-9c17-e46d6c2915d6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:47.869987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee570d34-bdd4-4f4c-9c17-e46d6c2915d6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:47.899144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ee570d34-bdd4-4f4c-9c17-e46d6c2915d6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044743 Aug 24 10:25:47.899687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 525/1187] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:47 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:47.903584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6de96b7b-232d-4790-989b-d2495a904842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:47.905724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6de96b7b-232d-4790-989b-d2495a904842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:47.912177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6de96b7b-232d-4790-989b-d2495a904842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:47.912662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6de96b7b-232d-4790-989b-d2495a904842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:47.913130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6de96b7b-232d-4790-989b-d2495a904842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:47.913817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6de96b7b-232d-4790-989b-d2495a904842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:47.914273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6de96b7b-232d-4790-989b-d2495a904842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:47.914695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6de96b7b-232d-4790-989b-d2495a904842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:47.944265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6de96b7b-232d-4790-989b-d2495a904842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044556 Aug 24 10:25:47.944805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 663/1188] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:47 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:48.312137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:48.312750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:48.318887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:48.319372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:48.319806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:48.320516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:48.320969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:48.321390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:48.381366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:48.388975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1a4e288b-7a95-4a64-9ab2-ef9fffed20e0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080867 Aug 24 10:25:48.389530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 526/1189] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:48 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:48.912301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-93b9843d-fc84-494c-b9c9-a83f319715e0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:48.914425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-93b9843d-fc84-494c-b9c9-a83f319715e0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:48.920891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93b9843d-fc84-494c-b9c9-a83f319715e0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:48.921377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93b9843d-fc84-494c-b9c9-a83f319715e0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:48.921821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93b9843d-fc84-494c-b9c9-a83f319715e0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:48.922521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93b9843d-fc84-494c-b9c9-a83f319715e0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:48.924852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93b9843d-fc84-494c-b9c9-a83f319715e0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:48.924852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93b9843d-fc84-494c-b9c9-a83f319715e0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:48.953309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-93b9843d-fc84-494c-b9c9-a83f319715e0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045698 Aug 24 10:25:48.953872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 664/1190] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:48 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:48.957277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2402f8a8-39f1-497b-a1df-8d9fec96aafd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:48.959441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2402f8a8-39f1-497b-a1df-8d9fec96aafd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:48.961757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=86 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:48.966840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2402f8a8-39f1-497b-a1df-8d9fec96aafd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:48.967320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2402f8a8-39f1-497b-a1df-8d9fec96aafd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:48.967746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2402f8a8-39f1-497b-a1df-8d9fec96aafd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:48.968459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2402f8a8-39f1-497b-a1df-8d9fec96aafd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:48.968908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2402f8a8-39f1-497b-a1df-8d9fec96aafd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:48.969328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2402f8a8-39f1-497b-a1df-8d9fec96aafd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:48.997915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2402f8a8-39f1-497b-a1df-8d9fec96aafd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045356 Aug 24 10:25:48.998455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 527/1191] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:48 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:49.410559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:49.411083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:49.416912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:49.417410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:49.417840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:49.418521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:49.418972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:49.419392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:49.480789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:49.489261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-df21abe3-c37a-41e2-9946-1e354aa2dfbe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083405 Aug 24 10:25:49.489810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 665/1192] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:49 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:49.966510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-692af877-ab76-4631-856e-73a9399e9a04 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:49.968701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-692af877-ab76-4631-856e-73a9399e9a04 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:49.975271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-692af877-ab76-4631-856e-73a9399e9a04 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:49.975749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-692af877-ab76-4631-856e-73a9399e9a04 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:49.976213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-692af877-ab76-4631-856e-73a9399e9a04 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:49.976891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-692af877-ab76-4631-856e-73a9399e9a04 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:49.977337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-692af877-ab76-4631-856e-73a9399e9a04 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:49.977755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-692af877-ab76-4631-856e-73a9399e9a04 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:49.981351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:50.006975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-692af877-ab76-4631-856e-73a9399e9a04 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044433 Aug 24 10:25:50.007522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 528/1193] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:49 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:50.009199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8aca0fe2-482a-4338-9e02-d425fb5a43d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:50.011339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8aca0fe2-482a-4338-9e02-d425fb5a43d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:50.013684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=86 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:50.019380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8aca0fe2-482a-4338-9e02-d425fb5a43d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:50.019871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8aca0fe2-482a-4338-9e02-d425fb5a43d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:50.020394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8aca0fe2-482a-4338-9e02-d425fb5a43d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:50.021052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8aca0fe2-482a-4338-9e02-d425fb5a43d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:50.021511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8aca0fe2-482a-4338-9e02-d425fb5a43d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:50.021941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8aca0fe2-482a-4338-9e02-d425fb5a43d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:50.052436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8aca0fe2-482a-4338-9e02-d425fb5a43d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047245 Aug 24 10:25:50.053004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 666/1194] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:50 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:50.509892 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:50.510503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:50.516384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:50.516863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:50.517287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:50.519605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:50.520088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:50.520511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:50.583794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:50.591648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-49dc8b90-6c26-4aff-9c52-046702010831 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085707 Aug 24 10:25:50.592262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 529/1195] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:50 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:51.018973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a5a3dff5-8444-457d-b859-2113a510ff49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:51.021186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a5a3dff5-8444-457d-b859-2113a510ff49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:51.027575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5a3dff5-8444-457d-b859-2113a510ff49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:51.028067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5a3dff5-8444-457d-b859-2113a510ff49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:51.028537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5a3dff5-8444-457d-b859-2113a510ff49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:51.029227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5a3dff5-8444-457d-b859-2113a510ff49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:51.029694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5a3dff5-8444-457d-b859-2113a510ff49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:51.030144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5a3dff5-8444-457d-b859-2113a510ff49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:51.059534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a5a3dff5-8444-457d-b859-2113a510ff49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044524 Aug 24 10:25:51.060274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 667/1196] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:51 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:51.065282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0351435c-7890-4349-9783-194fb82c41fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:51.068129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0351435c-7890-4349-9783-194fb82c41fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:51.074635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0351435c-7890-4349-9783-194fb82c41fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:51.075117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0351435c-7890-4349-9783-194fb82c41fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:51.075555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0351435c-7890-4349-9783-194fb82c41fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:51.076275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0351435c-7890-4349-9783-194fb82c41fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:51.076738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0351435c-7890-4349-9783-194fb82c41fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:51.077164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0351435c-7890-4349-9783-194fb82c41fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:51.106179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0351435c-7890-4349-9783-194fb82c41fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045393 Aug 24 10:25:51.106728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 530/1197] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:51 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:51.612477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:51.613096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:51.618973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:51.619452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:51.619889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:51.620607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:51.621068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:51.621492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:51.681740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:51.690341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e2126cdd-3ced-4682-a7ea-a0c542f19396 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081766 Aug 24 10:25:51.690904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 668/1198] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:51 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:52.072835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9b0cf228-3025-4c69-9477-1854850f5512 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:52.074978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9b0cf228-3025-4c69-9477-1854850f5512 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:52.081784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b0cf228-3025-4c69-9477-1854850f5512 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:52.082262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b0cf228-3025-4c69-9477-1854850f5512 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:52.082693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b0cf228-3025-4c69-9477-1854850f5512 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:52.083373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b0cf228-3025-4c69-9477-1854850f5512 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:52.083828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b0cf228-3025-4c69-9477-1854850f5512 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:52.084352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b0cf228-3025-4c69-9477-1854850f5512 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:52.114237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9b0cf228-3025-4c69-9477-1854850f5512 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045316 Aug 24 10:25:52.114795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 531/1199] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:52 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:52.118427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-38f6594c-84bf-4d5a-841a-037e8a38a869 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:52.120597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-38f6594c-84bf-4d5a-841a-037e8a38a869 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:52.127765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38f6594c-84bf-4d5a-841a-037e8a38a869 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:52.128296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38f6594c-84bf-4d5a-841a-037e8a38a869 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:52.128741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38f6594c-84bf-4d5a-841a-037e8a38a869 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:52.129441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38f6594c-84bf-4d5a-841a-037e8a38a869 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:52.129902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38f6594c-84bf-4d5a-841a-037e8a38a869 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:52.130331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38f6594c-84bf-4d5a-841a-037e8a38a869 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:52.159976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-38f6594c-84bf-4d5a-841a-037e8a38a869 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045566 Aug 24 10:25:52.160560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 669/1200] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:52 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:52.711084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:52.711691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:52.717562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:52.718038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:52.718635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:52.719150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:52.719593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:52.720008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:52.780334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:52.788539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d779c18a-c387-4386-bfc9-ae191fac210a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081306 Aug 24 10:25:52.789368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 532/1201] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:52 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:53.126769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-51d9f98d-a3c5-4466-bd3a-63d034431601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:53.129213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-51d9f98d-a3c5-4466-bd3a-63d034431601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:53.135567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51d9f98d-a3c5-4466-bd3a-63d034431601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:53.136154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51d9f98d-a3c5-4466-bd3a-63d034431601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:53.136590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51d9f98d-a3c5-4466-bd3a-63d034431601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:53.137284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51d9f98d-a3c5-4466-bd3a-63d034431601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:53.137735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51d9f98d-a3c5-4466-bd3a-63d034431601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:53.138158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51d9f98d-a3c5-4466-bd3a-63d034431601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:53.170446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-51d9f98d-a3c5-4466-bd3a-63d034431601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047970 Aug 24 10:25:53.171004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 670/1202] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:53 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:53.172662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-212214c6-ad77-4bf8-88b3-76911187b0fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:53.174841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-212214c6-ad77-4bf8-88b3-76911187b0fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:53.181322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-212214c6-ad77-4bf8-88b3-76911187b0fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:53.181810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-212214c6-ad77-4bf8-88b3-76911187b0fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:53.182252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-212214c6-ad77-4bf8-88b3-76911187b0fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:53.182944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-212214c6-ad77-4bf8-88b3-76911187b0fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:53.183397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-212214c6-ad77-4bf8-88b3-76911187b0fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:53.183823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-212214c6-ad77-4bf8-88b3-76911187b0fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:53.213578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-212214c6-ad77-4bf8-88b3-76911187b0fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044924 Aug 24 10:25:53.214160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 533/1203] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:53.813867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:53.815105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:53.821007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:53.821499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:53.821943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:53.822635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:53.823096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:53.823527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:53.887406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:53.895220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ccb745d6-17b9-4601-87ba-c47ce571301b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085330 Aug 24 10:25:53.895780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 671/1204] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:53 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:54.182153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ec47ec80-fb4f-4fa1-a3b1-dc5e7d0b3859 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:54.184341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ec47ec80-fb4f-4fa1-a3b1-dc5e7d0b3859 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:54.190803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec47ec80-fb4f-4fa1-a3b1-dc5e7d0b3859 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:54.191276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec47ec80-fb4f-4fa1-a3b1-dc5e7d0b3859 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:54.191704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec47ec80-fb4f-4fa1-a3b1-dc5e7d0b3859 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:54.192446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec47ec80-fb4f-4fa1-a3b1-dc5e7d0b3859 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:54.192894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec47ec80-fb4f-4fa1-a3b1-dc5e7d0b3859 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:54.193310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec47ec80-fb4f-4fa1-a3b1-dc5e7d0b3859 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:54.222281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ec47ec80-fb4f-4fa1-a3b1-dc5e7d0b3859 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044037 Aug 24 10:25:54.222830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 534/1205] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:54 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:54.226487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6cced250-60cc-4ca3-a37f-5a35e94a0325 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:54.228629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6cced250-60cc-4ca3-a37f-5a35e94a0325 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:54.235015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6cced250-60cc-4ca3-a37f-5a35e94a0325 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:54.235499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6cced250-60cc-4ca3-a37f-5a35e94a0325 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:54.235946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6cced250-60cc-4ca3-a37f-5a35e94a0325 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:54.237335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6cced250-60cc-4ca3-a37f-5a35e94a0325 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:54.237799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6cced250-60cc-4ca3-a37f-5a35e94a0325 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:54.238226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6cced250-60cc-4ca3-a37f-5a35e94a0325 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:54.266835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6cced250-60cc-4ca3-a37f-5a35e94a0325 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044257 Aug 24 10:25:54.267385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 672/1206] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:54 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:54.915412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:54.916016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:54.921938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:54.922414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:54.922841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:54.923515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:54.924012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:54.924451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:54.985988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:54.993967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4dda6bd7-13bf-4eb1-97a6-423813d13f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082440 Aug 24 10:25:54.994521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 535/1207] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:54 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:55.233737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6792a764-cd55-4c63-b1a6-8b92633048d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:55.235857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6792a764-cd55-4c63-b1a6-8b92633048d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:55.242227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6792a764-cd55-4c63-b1a6-8b92633048d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:55.242718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6792a764-cd55-4c63-b1a6-8b92633048d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:55.243161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6792a764-cd55-4c63-b1a6-8b92633048d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:55.243892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6792a764-cd55-4c63-b1a6-8b92633048d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:55.244381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6792a764-cd55-4c63-b1a6-8b92633048d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:55.244815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6792a764-cd55-4c63-b1a6-8b92633048d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:55.248372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:25:55.274547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6792a764-cd55-4c63-b1a6-8b92633048d8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044838 Aug 24 10:25:55.275094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 673/1208] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:55 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:55.279667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1e5d8795-ba88-423f-aadd-3a8c7761d252 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:55.281855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1e5d8795-ba88-423f-aadd-3a8c7761d252 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:55.288211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e5d8795-ba88-423f-aadd-3a8c7761d252 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:55.288687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e5d8795-ba88-423f-aadd-3a8c7761d252 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:55.289116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e5d8795-ba88-423f-aadd-3a8c7761d252 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:55.289799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e5d8795-ba88-423f-aadd-3a8c7761d252 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:55.290266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e5d8795-ba88-423f-aadd-3a8c7761d252 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:55.290699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e5d8795-ba88-423f-aadd-3a8c7761d252 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:55.320773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1e5d8795-ba88-423f-aadd-3a8c7761d252 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045071 Aug 24 10:25:55.321321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 536/1209] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:55 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:56.014669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:56.015309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:56.022058 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:56.022480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:56.022918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:56.023608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:56.024100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:56.024531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:56.085113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:56.092836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9b7e0a30-55f5-4437-8b7e-57beb6936ff3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082030 Aug 24 10:25:56.093390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 674/1210] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:56 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:56.286860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6abed2f8-f6d3-4ae7-b5cd-fc19810e90d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:56.288983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6abed2f8-f6d3-4ae7-b5cd-fc19810e90d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:56.295463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6abed2f8-f6d3-4ae7-b5cd-fc19810e90d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:56.295950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6abed2f8-f6d3-4ae7-b5cd-fc19810e90d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:56.296422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6abed2f8-f6d3-4ae7-b5cd-fc19810e90d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:56.297119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6abed2f8-f6d3-4ae7-b5cd-fc19810e90d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:56.297581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6abed2f8-f6d3-4ae7-b5cd-fc19810e90d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:56.298004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6abed2f8-f6d3-4ae7-b5cd-fc19810e90d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:56.327439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6abed2f8-f6d3-4ae7-b5cd-fc19810e90d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044696 Aug 24 10:25:56.328167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 537/1211] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:56 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:56.333418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b9adcb15-8022-4b22-b1b5-032837b06f28 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:56.335624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b9adcb15-8022-4b22-b1b5-032837b06f28 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:56.342956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9adcb15-8022-4b22-b1b5-032837b06f28 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:56.343616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9adcb15-8022-4b22-b1b5-032837b06f28 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:56.344215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9adcb15-8022-4b22-b1b5-032837b06f28 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:56.345005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9adcb15-8022-4b22-b1b5-032837b06f28 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:56.345460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9adcb15-8022-4b22-b1b5-032837b06f28 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:56.346065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9adcb15-8022-4b22-b1b5-032837b06f28 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:56.376665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b9adcb15-8022-4b22-b1b5-032837b06f28 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047519 Aug 24 10:25:56.377914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 675/1212] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:56 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:25:57.113208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:57.113727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:57.119581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:57.120093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:57.120522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:57.121194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:57.121644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:57.122057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:57.183627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:57.192397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-14751a89-9746-4516-acfd-5235df0365b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083165 Aug 24 10:25:57.193090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 538/1213] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:57 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:57.340172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0ebcc13b-89e4-40e9-976e-bb87c8f6eb2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:57.342307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0ebcc13b-89e4-40e9-976e-bb87c8f6eb2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:57.348797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ebcc13b-89e4-40e9-976e-bb87c8f6eb2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:57.349281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ebcc13b-89e4-40e9-976e-bb87c8f6eb2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:57.349725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ebcc13b-89e4-40e9-976e-bb87c8f6eb2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:57.350416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ebcc13b-89e4-40e9-976e-bb87c8f6eb2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:57.350872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ebcc13b-89e4-40e9-976e-bb87c8f6eb2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:57.351301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ebcc13b-89e4-40e9-976e-bb87c8f6eb2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:57.380352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0ebcc13b-89e4-40e9-976e-bb87c8f6eb2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044177 Aug 24 10:25:57.380823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 676/1214] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:57 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:57.389438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-300ab18f-6f30-4b67-92d9-6761e016e50a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:57.391611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-300ab18f-6f30-4b67-92d9-6761e016e50a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:57.398807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300ab18f-6f30-4b67-92d9-6761e016e50a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:57.399287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300ab18f-6f30-4b67-92d9-6761e016e50a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:57.399723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300ab18f-6f30-4b67-92d9-6761e016e50a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:57.400449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300ab18f-6f30-4b67-92d9-6761e016e50a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:57.400907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300ab18f-6f30-4b67-92d9-6761e016e50a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:57.401335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300ab18f-6f30-4b67-92d9-6761e016e50a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:57.430185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-300ab18f-6f30-4b67-92d9-6761e016e50a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044679 Aug 24 10:25:57.430735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 539/1215] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:57 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:58.213729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:58.214374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:58.220308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:58.221283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:58.221724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:58.222412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:58.222878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:58.223307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:58.282892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:58.291162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3d5aa0a9-ccbd-4ee4-a053-057bd4fac52b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081343 Aug 24 10:25:58.291717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 677/1216] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:58 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:25:58.393018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-48742a2c-b371-4274-8134-e7918e034415 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:58.395149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-48742a2c-b371-4274-8134-e7918e034415 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:58.401785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-48742a2c-b371-4274-8134-e7918e034415 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:58.402270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-48742a2c-b371-4274-8134-e7918e034415 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:58.402703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-48742a2c-b371-4274-8134-e7918e034415 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:58.403384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-48742a2c-b371-4274-8134-e7918e034415 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:58.403837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-48742a2c-b371-4274-8134-e7918e034415 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:58.404285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-48742a2c-b371-4274-8134-e7918e034415 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:58.433645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-48742a2c-b371-4274-8134-e7918e034415 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045049 Aug 24 10:25:58.434195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 540/1217] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:58 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:58.442300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d679462d-dd46-4be2-b5a0-23f7b03464f6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:58.444476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d679462d-dd46-4be2-b5a0-23f7b03464f6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:58.450836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d679462d-dd46-4be2-b5a0-23f7b03464f6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:58.451317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d679462d-dd46-4be2-b5a0-23f7b03464f6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:58.451746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d679462d-dd46-4be2-b5a0-23f7b03464f6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:58.452448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d679462d-dd46-4be2-b5a0-23f7b03464f6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:58.452904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d679462d-dd46-4be2-b5a0-23f7b03464f6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:58.453325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d679462d-dd46-4be2-b5a0-23f7b03464f6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:58.483381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d679462d-dd46-4be2-b5a0-23f7b03464f6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044842 Aug 24 10:25:58.483854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 678/1218] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:58 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:59.311869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:59.312492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:59.319299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:59.319837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:59.320402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:59.321127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:59.321580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:59.322002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:59.389391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:25:59.397285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-21c4d07f-09d1-413f-8ae1-ef91793c53e9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.089290 Aug 24 10:25:59.398163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 541/1219] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:25:59 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:25:59.446708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c018380f-28b6-41b6-b97f-38ceec25e116 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:59.448867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c018380f-28b6-41b6-b97f-38ceec25e116 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:59.456274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c018380f-28b6-41b6-b97f-38ceec25e116 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:59.456877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c018380f-28b6-41b6-b97f-38ceec25e116 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:59.457345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c018380f-28b6-41b6-b97f-38ceec25e116 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:59.458047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c018380f-28b6-41b6-b97f-38ceec25e116 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:59.458504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c018380f-28b6-41b6-b97f-38ceec25e116 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:59.458930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c018380f-28b6-41b6-b97f-38ceec25e116 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:59.488266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c018380f-28b6-41b6-b97f-38ceec25e116 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046427 Aug 24 10:25:59.489033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 679/1220] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:59 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:25:59.495420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e2cf4ac8-ad27-494e-b655-48228e5f5f86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:25:59.497580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e2cf4ac8-ad27-494e-b655-48228e5f5f86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:25:59.503887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2cf4ac8-ad27-494e-b655-48228e5f5f86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:59.504389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2cf4ac8-ad27-494e-b655-48228e5f5f86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:59.504818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2cf4ac8-ad27-494e-b655-48228e5f5f86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:59.505493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2cf4ac8-ad27-494e-b655-48228e5f5f86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:25:59.505941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2cf4ac8-ad27-494e-b655-48228e5f5f86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:25:59.506358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e2cf4ac8-ad27-494e-b655-48228e5f5f86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:25:59.535194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e2cf4ac8-ad27-494e-b655-48228e5f5f86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043662 Aug 24 10:25:59.535728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 542/1221] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:25:59 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:00.418498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:00.419100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:00.425061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:00.425548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:00.425980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:00.426664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:00.427126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:00.427537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:00.489660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:00.497477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7d169034-e88b-4912-9ac3-970f7a1f60a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082853 Aug 24 10:26:00.498029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 680/1222] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:00 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:00.500556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ec0f4f42-b093-4026-89b5-cd1635928165 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:00.503664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ec0f4f42-b093-4026-89b5-cd1635928165 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:00.510160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec0f4f42-b093-4026-89b5-cd1635928165 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:00.510645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec0f4f42-b093-4026-89b5-cd1635928165 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:00.511081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec0f4f42-b093-4026-89b5-cd1635928165 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:00.512176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec0f4f42-b093-4026-89b5-cd1635928165 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:00.512637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec0f4f42-b093-4026-89b5-cd1635928165 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:00.513066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec0f4f42-b093-4026-89b5-cd1635928165 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:00.542794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ec0f4f42-b093-4026-89b5-cd1635928165 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046093 Aug 24 10:26:00.543495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 543/1223] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:00 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:00.547252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0abc3248-3050-4af1-97b6-e8559eb7825c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:00.549932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0abc3248-3050-4af1-97b6-e8559eb7825c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:00.555976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0abc3248-3050-4af1-97b6-e8559eb7825c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:00.556614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0abc3248-3050-4af1-97b6-e8559eb7825c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:00.557132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0abc3248-3050-4af1-97b6-e8559eb7825c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:00.557932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0abc3248-3050-4af1-97b6-e8559eb7825c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:00.558422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0abc3248-3050-4af1-97b6-e8559eb7825c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:00.558974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0abc3248-3050-4af1-97b6-e8559eb7825c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:00.590586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0abc3248-3050-4af1-97b6-e8559eb7825c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047364 Aug 24 10:26:00.591135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 681/1224] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:00 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:01.520277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:01.520896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:01.527304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:01.527783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:01.528245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:01.528940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:01.529388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:01.529804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:01.554313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7caf3808-97c3-4e8d-999e-1dec6f33e336 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:01.556605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7caf3808-97c3-4e8d-999e-1dec6f33e336 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:01.564308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7caf3808-97c3-4e8d-999e-1dec6f33e336 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:01.565053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7caf3808-97c3-4e8d-999e-1dec6f33e336 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:01.565658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7caf3808-97c3-4e8d-999e-1dec6f33e336 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:01.566524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7caf3808-97c3-4e8d-999e-1dec6f33e336 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:01.567018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7caf3808-97c3-4e8d-999e-1dec6f33e336 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:01.567450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7caf3808-97c3-4e8d-999e-1dec6f33e336 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:01.598275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:01.601527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:01.603125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7caf3808-97c3-4e8d-999e-1dec6f33e336 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052519 Aug 24 10:26:01.603842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 682/1225] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:01 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:01.608872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-71c6cbfc-8728-4229-a380-2e5ffc83a595 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:01.611102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-71c6cbfc-8728-4229-a380-2e5ffc83a595 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:01.612357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e4f27dfe-1b2a-4d06-ab96-7fce66816f0b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.096821 Aug 24 10:26:01.612939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 544/1226] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:01 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:01.617058 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71c6cbfc-8728-4229-a380-2e5ffc83a595 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:01.617540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71c6cbfc-8728-4229-a380-2e5ffc83a595 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:01.617959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71c6cbfc-8728-4229-a380-2e5ffc83a595 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:01.618650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71c6cbfc-8728-4229-a380-2e5ffc83a595 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:01.619103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71c6cbfc-8728-4229-a380-2e5ffc83a595 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:01.619516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-71c6cbfc-8728-4229-a380-2e5ffc83a595 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:01.649697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-71c6cbfc-8728-4229-a380-2e5ffc83a595 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044774 Aug 24 10:26:01.650243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 683/1227] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:01 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:02.616155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b06335a1-3d0e-4f05-94b4-64efe7e89ba8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:02.618296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b06335a1-3d0e-4f05-94b4-64efe7e89ba8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:02.624745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b06335a1-3d0e-4f05-94b4-64efe7e89ba8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:02.625226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b06335a1-3d0e-4f05-94b4-64efe7e89ba8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:02.625663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b06335a1-3d0e-4f05-94b4-64efe7e89ba8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:02.626350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b06335a1-3d0e-4f05-94b4-64efe7e89ba8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:02.627370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b06335a1-3d0e-4f05-94b4-64efe7e89ba8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:02.627896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b06335a1-3d0e-4f05-94b4-64efe7e89ba8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:02.633458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:02.634070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:02.640342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:02.640823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:02.641258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:02.641946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:02.642982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:02.643405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:02.660213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b06335a1-3d0e-4f05-94b4-64efe7e89ba8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047922 Aug 24 10:26:02.660860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 545/1228] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:02 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:02.665924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-55e47184-b450-4bdd-80c1-296be46e6c23 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:02.668230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-55e47184-b450-4bdd-80c1-296be46e6c23 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:02.674343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e47184-b450-4bdd-80c1-296be46e6c23 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:02.674831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e47184-b450-4bdd-80c1-296be46e6c23 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:02.675267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e47184-b450-4bdd-80c1-296be46e6c23 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:02.675976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e47184-b450-4bdd-80c1-296be46e6c23 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:02.676445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e47184-b450-4bdd-80c1-296be46e6c23 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:02.676901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e47184-b450-4bdd-80c1-296be46e6c23 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:02.709448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-55e47184-b450-4bdd-80c1-296be46e6c23 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047504 Aug 24 10:26:02.710295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 546/1229] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:02 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:02.713167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:02.721571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-beb16ed7-8586-4fb8-9d40-be2033a26f80 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.091847 Aug 24 10:26:02.722131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 684/1230] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:02 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:03.673805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-71e05fe8-21ff-4c35-9c31-dd54a824e7b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:03.675941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-71e05fe8-21ff-4c35-9c31-dd54a824e7b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:03.682354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-71e05fe8-21ff-4c35-9c31-dd54a824e7b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:03.682856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-71e05fe8-21ff-4c35-9c31-dd54a824e7b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:03.683290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-71e05fe8-21ff-4c35-9c31-dd54a824e7b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:03.683976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-71e05fe8-21ff-4c35-9c31-dd54a824e7b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:03.684452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-71e05fe8-21ff-4c35-9c31-dd54a824e7b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:03.684876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-71e05fe8-21ff-4c35-9c31-dd54a824e7b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:03.715341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-71e05fe8-21ff-4c35-9c31-dd54a824e7b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045925 Aug 24 10:26:03.716017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 547/1231] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:03 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:03.721971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ddcf04d0-3bc8-448e-9e25-5ff99472587b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:03.724165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ddcf04d0-3bc8-448e-9e25-5ff99472587b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:03.730899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddcf04d0-3bc8-448e-9e25-5ff99472587b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:03.731388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddcf04d0-3bc8-448e-9e25-5ff99472587b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:03.731845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddcf04d0-3bc8-448e-9e25-5ff99472587b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:03.732632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddcf04d0-3bc8-448e-9e25-5ff99472587b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:03.733089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddcf04d0-3bc8-448e-9e25-5ff99472587b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:03.733519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddcf04d0-3bc8-448e-9e25-5ff99472587b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:03.743252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:03.743884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:03.750619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:03.751113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:03.751597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:03.752866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:03.753321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:03.753746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:03.765740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ddcf04d0-3bc8-448e-9e25-5ff99472587b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047778 Aug 24 10:26:03.766512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 685/1232] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:03 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:03.823679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:03.832022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-72d6c61b-6b01-471d-b8ab-e40d77846b4a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.092724 Aug 24 10:26:03.832636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 548/1233] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:03 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:04.728325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ba0bf948-e293-42f0-9f85-eaa706bc1381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:04.730568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ba0bf948-e293-42f0-9f85-eaa706bc1381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:04.737043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba0bf948-e293-42f0-9f85-eaa706bc1381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:04.737524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba0bf948-e293-42f0-9f85-eaa706bc1381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:04.737960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba0bf948-e293-42f0-9f85-eaa706bc1381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:04.738652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba0bf948-e293-42f0-9f85-eaa706bc1381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:04.739109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba0bf948-e293-42f0-9f85-eaa706bc1381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:04.739539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba0bf948-e293-42f0-9f85-eaa706bc1381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:04.769317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ba0bf948-e293-42f0-9f85-eaa706bc1381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044907 Aug 24 10:26:04.773234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 686/1234] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:04 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:04.778829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7337e1f9-ea43-448d-9398-420553b191e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:04.781030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7337e1f9-ea43-448d-9398-420553b191e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:04.787707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7337e1f9-ea43-448d-9398-420553b191e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:04.788250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7337e1f9-ea43-448d-9398-420553b191e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:04.788690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7337e1f9-ea43-448d-9398-420553b191e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:04.789398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7337e1f9-ea43-448d-9398-420553b191e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:04.789858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7337e1f9-ea43-448d-9398-420553b191e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:04.790285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7337e1f9-ea43-448d-9398-420553b191e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:04.821850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7337e1f9-ea43-448d-9398-420553b191e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047152 Aug 24 10:26:04.822411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 549/1235] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:04 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:04.852564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:04.853175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:04.859083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:04.859545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:04.859984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:04.860695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:04.861145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:04.861568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:04.922758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:04.930496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-21c4694d-07b3-49cb-90f3-1a240e344bcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081851 Aug 24 10:26:04.931047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 687/1236] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:05.781757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-abd9e47c-fa24-4be8-93ca-5fcf7029ca8d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:05.783921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-abd9e47c-fa24-4be8-93ca-5fcf7029ca8d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:05.790746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abd9e47c-fa24-4be8-93ca-5fcf7029ca8d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:05.791225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abd9e47c-fa24-4be8-93ca-5fcf7029ca8d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:05.791655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abd9e47c-fa24-4be8-93ca-5fcf7029ca8d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:05.792385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abd9e47c-fa24-4be8-93ca-5fcf7029ca8d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:05.792835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abd9e47c-fa24-4be8-93ca-5fcf7029ca8d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:05.793256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abd9e47c-fa24-4be8-93ca-5fcf7029ca8d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:05.824188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-abd9e47c-fa24-4be8-93ca-5fcf7029ca8d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046294 Aug 24 10:26:05.824743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 550/1237] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:05 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:05.834266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1727910b-0a4b-41ad-8e68-a486f493b84f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:05.836473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1727910b-0a4b-41ad-8e68-a486f493b84f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:05.842551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1727910b-0a4b-41ad-8e68-a486f493b84f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:05.843122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1727910b-0a4b-41ad-8e68-a486f493b84f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:05.843492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1727910b-0a4b-41ad-8e68-a486f493b84f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:05.844402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1727910b-0a4b-41ad-8e68-a486f493b84f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:05.844764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1727910b-0a4b-41ad-8e68-a486f493b84f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:05.845197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1727910b-0a4b-41ad-8e68-a486f493b84f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:05.874529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1727910b-0a4b-41ad-8e68-a486f493b84f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044182 Aug 24 10:26:05.875077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 688/1238] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:05 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:05.952494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:05.953108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:05.959424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:05.959916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:05.960384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:05.961082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:05.961542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:05.961966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:06.029926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:06.038024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5a28e7fe-099f-4ea7-b7b9-29192cfaeaca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.089366 Aug 24 10:26:06.038606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 551/1239] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:06.837120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1c676c2f-7104-4e11-814b-83e4eebda033 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:06.839268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1c676c2f-7104-4e11-814b-83e4eebda033 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:06.846450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c676c2f-7104-4e11-814b-83e4eebda033 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:06.846934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c676c2f-7104-4e11-814b-83e4eebda033 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:06.847376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c676c2f-7104-4e11-814b-83e4eebda033 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:06.848100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c676c2f-7104-4e11-814b-83e4eebda033 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:06.848564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c676c2f-7104-4e11-814b-83e4eebda033 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:06.848997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c676c2f-7104-4e11-814b-83e4eebda033 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:06.873658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:06.878737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1c676c2f-7104-4e11-814b-83e4eebda033 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045662 Aug 24 10:26:06.879294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 689/1240] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:06 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:06.887565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-192899e5-53b0-4f74-9aaa-ce8ef8c84c18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:06.890134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-192899e5-53b0-4f74-9aaa-ce8ef8c84c18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:06.898079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-192899e5-53b0-4f74-9aaa-ce8ef8c84c18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:06.898634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-192899e5-53b0-4f74-9aaa-ce8ef8c84c18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:06.899113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-192899e5-53b0-4f74-9aaa-ce8ef8c84c18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:06.899888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-192899e5-53b0-4f74-9aaa-ce8ef8c84c18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:06.900458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-192899e5-53b0-4f74-9aaa-ce8ef8c84c18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:06.900995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-192899e5-53b0-4f74-9aaa-ce8ef8c84c18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:06.931387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-192899e5-53b0-4f74-9aaa-ce8ef8c84c18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048484 Aug 24 10:26:06.931937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 552/1241] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:06 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:07.059338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:07.059947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:07.066596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:07.067076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:07.067510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:07.068236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:07.068688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:07.069107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:07.129057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:07.136687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3e305330-6881-4c2c-89b5-6f2d91a401dd tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081250 Aug 24 10:26:07.137242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 690/1242] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:07 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:07.890983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ebd53aee-3b2f-4bfa-8ee3-c0bce2936abd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:07.893274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ebd53aee-3b2f-4bfa-8ee3-c0bce2936abd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:07.899638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd53aee-3b2f-4bfa-8ee3-c0bce2936abd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:07.900156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd53aee-3b2f-4bfa-8ee3-c0bce2936abd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:07.900595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd53aee-3b2f-4bfa-8ee3-c0bce2936abd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:07.901286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd53aee-3b2f-4bfa-8ee3-c0bce2936abd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:07.901774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd53aee-3b2f-4bfa-8ee3-c0bce2936abd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:07.902201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd53aee-3b2f-4bfa-8ee3-c0bce2936abd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:07.932060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ebd53aee-3b2f-4bfa-8ee3-c0bce2936abd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045025 Aug 24 10:26:07.932632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 553/1243] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:07 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:07.942908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fc74f24f-5c1d-436f-98b7-6acc71bbaf2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:07.945044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fc74f24f-5c1d-436f-98b7-6acc71bbaf2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:07.951288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc74f24f-5c1d-436f-98b7-6acc71bbaf2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:07.951774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc74f24f-5c1d-436f-98b7-6acc71bbaf2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:07.952245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc74f24f-5c1d-436f-98b7-6acc71bbaf2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:07.952936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc74f24f-5c1d-436f-98b7-6acc71bbaf2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:07.953387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc74f24f-5c1d-436f-98b7-6acc71bbaf2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:07.954059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc74f24f-5c1d-436f-98b7-6acc71bbaf2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:07.984354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fc74f24f-5c1d-436f-98b7-6acc71bbaf2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045452 Aug 24 10:26:07.984915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 691/1244] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:07 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:08.158839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:08.159471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:08.165371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:08.165853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:08.166289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:08.166980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:08.167437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:08.167856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:08.233166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:08.242147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a0e52b8b-ef96-4314-83d1-f9025ef317b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087961 Aug 24 10:26:08.242709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 554/1245] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:08 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:08.944416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d04d58bc-20f8-443b-9589-5706b865f5f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:08.946531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d04d58bc-20f8-443b-9589-5706b865f5f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:08.952922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d04d58bc-20f8-443b-9589-5706b865f5f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:08.953401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d04d58bc-20f8-443b-9589-5706b865f5f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:08.954094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d04d58bc-20f8-443b-9589-5706b865f5f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:08.955275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d04d58bc-20f8-443b-9589-5706b865f5f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:08.955728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d04d58bc-20f8-443b-9589-5706b865f5f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:08.956186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d04d58bc-20f8-443b-9589-5706b865f5f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:08.985166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d04d58bc-20f8-443b-9589-5706b865f5f1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044719 Aug 24 10:26:08.985738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 692/1246] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:08.997010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ed5b4d00-8030-454e-b5db-14c40e933b32 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:08.999166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ed5b4d00-8030-454e-b5db-14c40e933b32 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:09.005623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed5b4d00-8030-454e-b5db-14c40e933b32 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:09.006094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed5b4d00-8030-454e-b5db-14c40e933b32 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:09.006519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed5b4d00-8030-454e-b5db-14c40e933b32 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:09.007201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed5b4d00-8030-454e-b5db-14c40e933b32 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:09.007646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed5b4d00-8030-454e-b5db-14c40e933b32 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:09.008084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed5b4d00-8030-454e-b5db-14c40e933b32 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:09.037632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ed5b4d00-8030-454e-b5db-14c40e933b32 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044440 Aug 24 10:26:09.038111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 555/1247] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:08 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:09.263418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:09.264081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:09.271466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:09.271956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:09.272416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:09.273180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:09.273554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:09.274132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:09.339447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:09.347744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-91b73ca6-f883-45b0-a495-6ae421ce580d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.088829 Aug 24 10:26:09.356419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 693/1248] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:09 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:09.997391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-087aad47-95fa-4ad9-8542-a7dc15670e38 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:09.999535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-087aad47-95fa-4ad9-8542-a7dc15670e38 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:10.005614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-087aad47-95fa-4ad9-8542-a7dc15670e38 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:10.006088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-087aad47-95fa-4ad9-8542-a7dc15670e38 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:10.006518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-087aad47-95fa-4ad9-8542-a7dc15670e38 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:10.007197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-087aad47-95fa-4ad9-8542-a7dc15670e38 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:10.007643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-087aad47-95fa-4ad9-8542-a7dc15670e38 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:10.008080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-087aad47-95fa-4ad9-8542-a7dc15670e38 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:10.036864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-087aad47-95fa-4ad9-8542-a7dc15670e38 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043348 Aug 24 10:26:10.037332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 556/1249] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:09 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:10.050423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-716bcb0c-d1ef-4fd2-9132-8c43d43a4467 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:10.052566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-716bcb0c-d1ef-4fd2-9132-8c43d43a4467 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:10.058973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-716bcb0c-d1ef-4fd2-9132-8c43d43a4467 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:10.059473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-716bcb0c-d1ef-4fd2-9132-8c43d43a4467 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:10.059909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-716bcb0c-d1ef-4fd2-9132-8c43d43a4467 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:10.060629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-716bcb0c-d1ef-4fd2-9132-8c43d43a4467 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:10.061082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-716bcb0c-d1ef-4fd2-9132-8c43d43a4467 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:10.061507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-716bcb0c-d1ef-4fd2-9132-8c43d43a4467 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:10.091216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-716bcb0c-d1ef-4fd2-9132-8c43d43a4467 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045307 Aug 24 10:26:10.091778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 694/1250] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:10 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:10.367733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:10.368291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:10.374417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:10.374894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:10.375326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:10.376008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:10.377125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:10.377545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:10.436896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:10.446909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-96d510a2-cbec-49c5-a4a4-d605204755de tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083155 Aug 24 10:26:10.447467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 557/1251] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:10 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:11.049204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2e3271a2-ea16-4fd5-beb0-2186de4b47bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:11.051225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2e3271a2-ea16-4fd5-beb0-2186de4b47bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:11.057691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e3271a2-ea16-4fd5-beb0-2186de4b47bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:11.058168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e3271a2-ea16-4fd5-beb0-2186de4b47bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:11.058606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e3271a2-ea16-4fd5-beb0-2186de4b47bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:11.059293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e3271a2-ea16-4fd5-beb0-2186de4b47bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:11.059741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e3271a2-ea16-4fd5-beb0-2186de4b47bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:11.060263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e3271a2-ea16-4fd5-beb0-2186de4b47bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:11.090060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2e3271a2-ea16-4fd5-beb0-2186de4b47bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044841 Aug 24 10:26:11.090609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 695/1252] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:11 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:11.102912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6b6ee4ca-afce-4f94-aeb8-9c53e4f8ec55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:11.105105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6b6ee4ca-afce-4f94-aeb8-9c53e4f8ec55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:11.112186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b6ee4ca-afce-4f94-aeb8-9c53e4f8ec55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:11.112682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b6ee4ca-afce-4f94-aeb8-9c53e4f8ec55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:11.113118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b6ee4ca-afce-4f94-aeb8-9c53e4f8ec55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:11.113830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b6ee4ca-afce-4f94-aeb8-9c53e4f8ec55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:11.114287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b6ee4ca-afce-4f94-aeb8-9c53e4f8ec55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:11.114708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b6ee4ca-afce-4f94-aeb8-9c53e4f8ec55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:11.145665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6b6ee4ca-afce-4f94-aeb8-9c53e4f8ec55 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046747 Aug 24 10:26:11.146212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 558/1253] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:11 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:11.467520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:11.468186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:11.474163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:11.474650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:11.475095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:11.475818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:11.476318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:11.476747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:11.748368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:11.756855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a5aaf8d2-30e1-4804-941b-849d7d7f889e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.293129 Aug 24 10:26:11.757413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 696/1254] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:11 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:12.102827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3eaf29d9-239c-44f3-b3d5-f5f4cadfffb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:12.104968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3eaf29d9-239c-44f3-b3d5-f5f4cadfffb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:12.111814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3eaf29d9-239c-44f3-b3d5-f5f4cadfffb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:12.112323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3eaf29d9-239c-44f3-b3d5-f5f4cadfffb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:12.112756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3eaf29d9-239c-44f3-b3d5-f5f4cadfffb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:12.113447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3eaf29d9-239c-44f3-b3d5-f5f4cadfffb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:12.113909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3eaf29d9-239c-44f3-b3d5-f5f4cadfffb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:12.114322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3eaf29d9-239c-44f3-b3d5-f5f4cadfffb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:12.146407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3eaf29d9-239c-44f3-b3d5-f5f4cadfffb4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047544 Aug 24 10:26:12.146984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 559/1255] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:12.158869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-75299725-e98b-48e1-b4e9-fe682e0e8e4e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:12.161078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-75299725-e98b-48e1-b4e9-fe682e0e8e4e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:12.167894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75299725-e98b-48e1-b4e9-fe682e0e8e4e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:12.168426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75299725-e98b-48e1-b4e9-fe682e0e8e4e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:12.168873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75299725-e98b-48e1-b4e9-fe682e0e8e4e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:12.169581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75299725-e98b-48e1-b4e9-fe682e0e8e4e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:12.170043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75299725-e98b-48e1-b4e9-fe682e0e8e4e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:12.170473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75299725-e98b-48e1-b4e9-fe682e0e8e4e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:12.202425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-75299725-e98b-48e1-b4e9-fe682e0e8e4e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048035 Aug 24 10:26:12.203015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 697/1256] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:12 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:12.777737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:12.778346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:12.785347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:12.786075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:12.786749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:12.787797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:12.788510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:12.789148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:12.851329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:12.859137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c9f20a9e-36aa-41f4-a529-0e26084ddca0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085306 Aug 24 10:26:12.859700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 560/1257] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:12 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:13.158931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0c63510d-4109-4c0f-9283-90cb4be45081 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:13.161051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0c63510d-4109-4c0f-9283-90cb4be45081 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:13.167096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c63510d-4109-4c0f-9283-90cb4be45081 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:13.167587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c63510d-4109-4c0f-9283-90cb4be45081 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:13.168031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c63510d-4109-4c0f-9283-90cb4be45081 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:13.168750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c63510d-4109-4c0f-9283-90cb4be45081 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:13.169216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c63510d-4109-4c0f-9283-90cb4be45081 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:13.169650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c63510d-4109-4c0f-9283-90cb4be45081 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:13.198975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0c63510d-4109-4c0f-9283-90cb4be45081 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044157 Aug 24 10:26:13.199433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 698/1258] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:13 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:13.214762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c2186629-1631-4039-88fa-8307f555145b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:13.216939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c2186629-1631-4039-88fa-8307f555145b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:13.223313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2186629-1631-4039-88fa-8307f555145b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:13.223792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2186629-1631-4039-88fa-8307f555145b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:13.224273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2186629-1631-4039-88fa-8307f555145b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:13.224964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2186629-1631-4039-88fa-8307f555145b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:13.225418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2186629-1631-4039-88fa-8307f555145b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:13.225844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2186629-1631-4039-88fa-8307f555145b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:13.242379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:13.256611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c2186629-1631-4039-88fa-8307f555145b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045749 Aug 24 10:26:13.257175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 561/1259] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:13 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:13.879237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:13.879841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:13.885721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:13.886198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:13.886635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:13.887324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:13.887773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:13.888225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:13.947578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:13.956447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5fab4c2c-93bc-4cec-87e6-e840f734cc40 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081506 Aug 24 10:26:13.957005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 699/1260] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:13 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:14.211342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-49d307c6-220b-47c0-ad84-04f357cbc739 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:14.213518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-49d307c6-220b-47c0-ad84-04f357cbc739 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:14.219493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49d307c6-220b-47c0-ad84-04f357cbc739 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:14.220704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49d307c6-220b-47c0-ad84-04f357cbc739 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:14.220965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49d307c6-220b-47c0-ad84-04f357cbc739 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:14.221674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49d307c6-220b-47c0-ad84-04f357cbc739 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:14.222120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49d307c6-220b-47c0-ad84-04f357cbc739 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:14.222534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49d307c6-220b-47c0-ad84-04f357cbc739 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:14.251268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-49d307c6-220b-47c0-ad84-04f357cbc739 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043890 Aug 24 10:26:14.251806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 562/1261] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:14 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:14.269682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-392e0dd0-b802-4e81-b27f-7a13272055e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:14.271842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-392e0dd0-b802-4e81-b27f-7a13272055e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:14.277679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392e0dd0-b802-4e81-b27f-7a13272055e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:14.278159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392e0dd0-b802-4e81-b27f-7a13272055e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:14.278594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392e0dd0-b802-4e81-b27f-7a13272055e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:14.279276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392e0dd0-b802-4e81-b27f-7a13272055e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:14.279726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392e0dd0-b802-4e81-b27f-7a13272055e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:14.280266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392e0dd0-b802-4e81-b27f-7a13272055e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:14.308842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-392e0dd0-b802-4e81-b27f-7a13272055e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044536 Aug 24 10:26:14.309391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 700/1262] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:14 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:14.978335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:14.978981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:14.984947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:14.985953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:14.986390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:14.987077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:14.987532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:14.987956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:15.048156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:15.055855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c05e734a-c9f0-4c6a-8010-2bb254901444 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081407 Aug 24 10:26:15.056445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 563/1263] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:14 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:15.262830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dfc842d0-7128-4857-b134-363648051062 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:15.265012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dfc842d0-7128-4857-b134-363648051062 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:15.272263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc842d0-7128-4857-b134-363648051062 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:15.272890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc842d0-7128-4857-b134-363648051062 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:15.273166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc842d0-7128-4857-b134-363648051062 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:15.273855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc842d0-7128-4857-b134-363648051062 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:15.274304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc842d0-7128-4857-b134-363648051062 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:15.274725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc842d0-7128-4857-b134-363648051062 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:15.303869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dfc842d0-7128-4857-b134-363648051062 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044878 Aug 24 10:26:15.304371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 701/1264] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:15 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:15.321139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b52c08ce-7d62-423c-b457-ef9bc3a401fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:15.323321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b52c08ce-7d62-423c-b457-ef9bc3a401fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:15.330288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b52c08ce-7d62-423c-b457-ef9bc3a401fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:15.330769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b52c08ce-7d62-423c-b457-ef9bc3a401fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:15.331204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b52c08ce-7d62-423c-b457-ef9bc3a401fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:15.331891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b52c08ce-7d62-423c-b457-ef9bc3a401fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:15.332378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b52c08ce-7d62-423c-b457-ef9bc3a401fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:15.332799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b52c08ce-7d62-423c-b457-ef9bc3a401fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:15.362263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b52c08ce-7d62-423c-b457-ef9bc3a401fe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045030 Aug 24 10:26:15.362814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 564/1265] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:15 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:16.077008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:16.077533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:16.084241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:16.084719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:16.085155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:16.085831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:16.086281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:16.086701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:16.147424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:16.155189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-214cd51a-a781-4369-9930-02f52c472377 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082314 Aug 24 10:26:16.155745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 702/1266] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:16 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:16.316816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-374a8dd0-0346-40c9-be0c-980207448fe1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:16.318946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-374a8dd0-0346-40c9-be0c-980207448fe1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:16.324855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-374a8dd0-0346-40c9-be0c-980207448fe1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:16.325416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-374a8dd0-0346-40c9-be0c-980207448fe1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:16.325749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-374a8dd0-0346-40c9-be0c-980207448fe1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:16.326429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-374a8dd0-0346-40c9-be0c-980207448fe1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:16.326874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-374a8dd0-0346-40c9-be0c-980207448fe1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:16.327345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-374a8dd0-0346-40c9-be0c-980207448fe1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:16.358413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-374a8dd0-0346-40c9-be0c-980207448fe1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046080 Aug 24 10:26:16.358959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 565/1267] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:16 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:16.374580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ea05d66d-7f9a-4428-a2f4-4c36d9d9be06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:16.376765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ea05d66d-7f9a-4428-a2f4-4c36d9d9be06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:16.383162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ea05d66d-7f9a-4428-a2f4-4c36d9d9be06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:16.383650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ea05d66d-7f9a-4428-a2f4-4c36d9d9be06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:16.384116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ea05d66d-7f9a-4428-a2f4-4c36d9d9be06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:16.384814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ea05d66d-7f9a-4428-a2f4-4c36d9d9be06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:16.385272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ea05d66d-7f9a-4428-a2f4-4c36d9d9be06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:16.385703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ea05d66d-7f9a-4428-a2f4-4c36d9d9be06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:16.415491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ea05d66d-7f9a-4428-a2f4-4c36d9d9be06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044919 Aug 24 10:26:16.416054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 703/1268] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:16 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:17.177156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:17.177690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:17.183657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:17.184170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:17.184609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:17.185303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:17.186026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:17.186482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:17.250024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:17.258410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4e90545e-5848-472e-a28b-271d990c46f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085778 Aug 24 10:26:17.258967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 566/1269] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:17 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:17.370003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b0cd165b-35ea-4c77-8119-916b98b56181 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:17.372122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b0cd165b-35ea-4c77-8119-916b98b56181 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:17.378669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0cd165b-35ea-4c77-8119-916b98b56181 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:17.379258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0cd165b-35ea-4c77-8119-916b98b56181 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:17.379568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0cd165b-35ea-4c77-8119-916b98b56181 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:17.380288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0cd165b-35ea-4c77-8119-916b98b56181 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:17.380769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0cd165b-35ea-4c77-8119-916b98b56181 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:17.381192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0cd165b-35ea-4c77-8119-916b98b56181 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:17.410138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b0cd165b-35ea-4c77-8119-916b98b56181 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044042 Aug 24 10:26:17.410674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 704/1270] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:17 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:17.427965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-353b902d-b2ff-4ad8-99cb-6f0bbbe12168 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:17.430142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-353b902d-b2ff-4ad8-99cb-6f0bbbe12168 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:17.436561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-353b902d-b2ff-4ad8-99cb-6f0bbbe12168 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:17.437040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-353b902d-b2ff-4ad8-99cb-6f0bbbe12168 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:17.437473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-353b902d-b2ff-4ad8-99cb-6f0bbbe12168 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:17.438815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-353b902d-b2ff-4ad8-99cb-6f0bbbe12168 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:17.439273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-353b902d-b2ff-4ad8-99cb-6f0bbbe12168 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:17.439697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-353b902d-b2ff-4ad8-99cb-6f0bbbe12168 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:17.468297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-353b902d-b2ff-4ad8-99cb-6f0bbbe12168 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044614 Aug 24 10:26:17.468844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 567/1271] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:17 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:18.278935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:18.279739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:18.286187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:18.286709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:18.287151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:18.287844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:18.288327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:18.288756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:18.349171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:18.356829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-022cc6a7-9d47-4388-b0df-388df1a94f37 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081771 Aug 24 10:26:18.357758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 705/1272] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:18 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:18.422330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c9bd39e2-78e9-420a-9920-89fa448a19bb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:18.425074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c9bd39e2-78e9-420a-9920-89fa448a19bb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:18.430998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9bd39e2-78e9-420a-9920-89fa448a19bb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:18.431608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9bd39e2-78e9-420a-9920-89fa448a19bb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:18.431942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9bd39e2-78e9-420a-9920-89fa448a19bb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:18.432645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9bd39e2-78e9-420a-9920-89fa448a19bb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:18.433097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9bd39e2-78e9-420a-9920-89fa448a19bb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:18.433521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9bd39e2-78e9-420a-9920-89fa448a19bb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:18.462745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c9bd39e2-78e9-420a-9920-89fa448a19bb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044340 Aug 24 10:26:18.463293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 568/1273] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:18 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:18.481265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ecedd946-3c33-4f89-8e4d-bb9afcca55b5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:18.483466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ecedd946-3c33-4f89-8e4d-bb9afcca55b5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:18.490488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecedd946-3c33-4f89-8e4d-bb9afcca55b5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:18.490975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecedd946-3c33-4f89-8e4d-bb9afcca55b5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:18.491415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecedd946-3c33-4f89-8e4d-bb9afcca55b5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:18.492131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecedd946-3c33-4f89-8e4d-bb9afcca55b5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:18.492591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecedd946-3c33-4f89-8e4d-bb9afcca55b5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:18.493021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecedd946-3c33-4f89-8e4d-bb9afcca55b5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:18.509583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:18.523525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ecedd946-3c33-4f89-8e4d-bb9afcca55b5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046182 Aug 24 10:26:18.524097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 706/1274] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:18 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:19.378340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:19.378954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:19.381025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=84 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:19.386367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:19.386847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:19.387285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:19.387975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:19.388509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:19.388509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:19.450054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:19.458383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6f12a8bb-bd07-48e9-8381-4902f2d4c29f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084164 Aug 24 10:26:19.458942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 569/1275] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:19 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:19.475412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f3b75559-b729-48fb-9e4e-2dbee4f7492d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:19.477590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f3b75559-b729-48fb-9e4e-2dbee4f7492d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:19.483442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3b75559-b729-48fb-9e4e-2dbee4f7492d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:19.483995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3b75559-b729-48fb-9e4e-2dbee4f7492d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:19.484396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3b75559-b729-48fb-9e4e-2dbee4f7492d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:19.485116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3b75559-b729-48fb-9e4e-2dbee4f7492d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:19.485570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3b75559-b729-48fb-9e4e-2dbee4f7492d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:19.485997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3b75559-b729-48fb-9e4e-2dbee4f7492d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:19.514786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f3b75559-b729-48fb-9e4e-2dbee4f7492d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043287 Aug 24 10:26:19.515330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 707/1276] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:19 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:19.535446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9acd456d-923b-44d0-aebf-e69fa3790eda tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:19.537673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9acd456d-923b-44d0-aebf-e69fa3790eda tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:19.543593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9acd456d-923b-44d0-aebf-e69fa3790eda tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:19.544095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9acd456d-923b-44d0-aebf-e69fa3790eda tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:19.544529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9acd456d-923b-44d0-aebf-e69fa3790eda tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:19.545217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9acd456d-923b-44d0-aebf-e69fa3790eda tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:19.545665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9acd456d-923b-44d0-aebf-e69fa3790eda tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:19.546081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9acd456d-923b-44d0-aebf-e69fa3790eda tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:19.574791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9acd456d-923b-44d0-aebf-e69fa3790eda tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043949 Aug 24 10:26:19.575336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 570/1277] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:19 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:20.479306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:20.479924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:20.482281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=84 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:20.488450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:20.489077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:20.489362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:20.490057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:20.490514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:20.490939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:20.526881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b6753e9d-17e4-40be-ae54-a20c1764561b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:20.529047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b6753e9d-17e4-40be-ae54-a20c1764561b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:20.536026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6753e9d-17e4-40be-ae54-a20c1764561b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:20.536546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6753e9d-17e4-40be-ae54-a20c1764561b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:20.536980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6753e9d-17e4-40be-ae54-a20c1764561b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:20.537683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6753e9d-17e4-40be-ae54-a20c1764561b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:20.538136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6753e9d-17e4-40be-ae54-a20c1764561b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:20.538560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6753e9d-17e4-40be-ae54-a20c1764561b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:20.555030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:20.562784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-52ebdb39-86ad-4c1f-ad77-7dc262801272 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087291 Aug 24 10:26:20.563360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 708/1278] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:20 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:20.569080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b6753e9d-17e4-40be-ae54-a20c1764561b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046147 Aug 24 10:26:20.569637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 571/1279] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:20 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:20.586635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6e05d896-91d8-44a9-9761-4b475e49b0c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:20.588780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6e05d896-91d8-44a9-9761-4b475e49b0c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:20.595168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e05d896-91d8-44a9-9761-4b475e49b0c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:20.595652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e05d896-91d8-44a9-9761-4b475e49b0c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:20.596109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e05d896-91d8-44a9-9761-4b475e49b0c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:20.596803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e05d896-91d8-44a9-9761-4b475e49b0c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:20.597260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e05d896-91d8-44a9-9761-4b475e49b0c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:20.597688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e05d896-91d8-44a9-9761-4b475e49b0c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:20.626174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6e05d896-91d8-44a9-9761-4b475e49b0c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043426 Aug 24 10:26:20.626728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 709/1280] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:20 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:21.580901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-57a92ebc-e802-4cc7-a214-1ace44deffbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:21.582666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:21.583060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-57a92ebc-e802-4cc7-a214-1ace44deffbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:21.583273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:21.589536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57a92ebc-e802-4cc7-a214-1ace44deffbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:21.589740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:21.590019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57a92ebc-e802-4cc7-a214-1ace44deffbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:21.590187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:21.590449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57a92ebc-e802-4cc7-a214-1ace44deffbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:21.590662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:21.591136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57a92ebc-e802-4cc7-a214-1ace44deffbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:21.591474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:21.591648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57a92ebc-e802-4cc7-a214-1ace44deffbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:21.591870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:21.591987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57a92ebc-e802-4cc7-a214-1ace44deffbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:21.592461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:21.621582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-57a92ebc-e802-4cc7-a214-1ace44deffbe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045102 Aug 24 10:26:21.622217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 572/1281] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:21 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:21.637679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f3d26d15-c578-4c8b-ba9b-31654198b409 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:21.639922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f3d26d15-c578-4c8b-ba9b-31654198b409 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:21.645955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3d26d15-c578-4c8b-ba9b-31654198b409 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:21.646440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3d26d15-c578-4c8b-ba9b-31654198b409 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:21.646873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3d26d15-c578-4c8b-ba9b-31654198b409 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:21.647562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3d26d15-c578-4c8b-ba9b-31654198b409 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:21.648011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3d26d15-c578-4c8b-ba9b-31654198b409 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:21.648459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3d26d15-c578-4c8b-ba9b-31654198b409 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:21.654021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:21.661828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ff68f1bc-dae0-488c-8290-ce1973967bb8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082872 Aug 24 10:26:21.662402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 710/1282] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:21 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:21.679874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f3d26d15-c578-4c8b-ba9b-31654198b409 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046102 Aug 24 10:26:21.680459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 573/1283] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:21 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:22.634933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1343b941-3d7e-4ad0-bb36-bec7e2ac75b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:22.637117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1343b941-3d7e-4ad0-bb36-bec7e2ac75b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:22.643529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1343b941-3d7e-4ad0-bb36-bec7e2ac75b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:22.644014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1343b941-3d7e-4ad0-bb36-bec7e2ac75b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:22.644496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1343b941-3d7e-4ad0-bb36-bec7e2ac75b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:22.645191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1343b941-3d7e-4ad0-bb36-bec7e2ac75b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:22.645643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1343b941-3d7e-4ad0-bb36-bec7e2ac75b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:22.646072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1343b941-3d7e-4ad0-bb36-bec7e2ac75b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:22.675719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1343b941-3d7e-4ad0-bb36-bec7e2ac75b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044838 Aug 24 10:26:22.676299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 711/1284] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:22 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:22.682197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:22.682807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:22.689668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:22.690144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:22.690576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:22.691260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:22.691701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3561e8c2-1c4c-41fe-8420-b3f1deaaa820 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:22.691822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:22.692147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:22.693839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3561e8c2-1c4c-41fe-8420-b3f1deaaa820 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:22.699620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3561e8c2-1c4c-41fe-8420-b3f1deaaa820 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:22.700146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3561e8c2-1c4c-41fe-8420-b3f1deaaa820 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:22.700587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3561e8c2-1c4c-41fe-8420-b3f1deaaa820 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:22.701563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3561e8c2-1c4c-41fe-8420-b3f1deaaa820 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:22.702022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3561e8c2-1c4c-41fe-8420-b3f1deaaa820 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:22.702447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3561e8c2-1c4c-41fe-8420-b3f1deaaa820 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:22.732853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3561e8c2-1c4c-41fe-8420-b3f1deaaa820 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044874 Aug 24 10:26:22.733414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 712/1285] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:22 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:22.754959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:22.762753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-72b8e20a-1a07-481b-a02d-86ab481a0558 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084364 Aug 24 10:26:22.763356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 574/1286] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:22 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:23.689178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b424a7ff-0c88-4d50-9ad4-f12823f1238f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:23.691293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b424a7ff-0c88-4d50-9ad4-f12823f1238f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:23.697627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b424a7ff-0c88-4d50-9ad4-f12823f1238f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:23.698100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b424a7ff-0c88-4d50-9ad4-f12823f1238f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:23.698533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b424a7ff-0c88-4d50-9ad4-f12823f1238f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:23.699214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b424a7ff-0c88-4d50-9ad4-f12823f1238f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:23.699664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b424a7ff-0c88-4d50-9ad4-f12823f1238f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:23.700106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b424a7ff-0c88-4d50-9ad4-f12823f1238f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:23.730063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b424a7ff-0c88-4d50-9ad4-f12823f1238f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045626 Aug 24 10:26:23.730620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 713/1287] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:23 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:23.745615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0a645658-0400-4018-8a3f-755e4180c066 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:23.747757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0a645658-0400-4018-8a3f-755e4180c066 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:23.754669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a645658-0400-4018-8a3f-755e4180c066 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:23.755151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a645658-0400-4018-8a3f-755e4180c066 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:23.755584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a645658-0400-4018-8a3f-755e4180c066 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:23.756298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a645658-0400-4018-8a3f-755e4180c066 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:23.756753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a645658-0400-4018-8a3f-755e4180c066 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:23.757174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a645658-0400-4018-8a3f-755e4180c066 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:23.782704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:23.783331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:23.786764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0a645658-0400-4018-8a3f-755e4180c066 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045039 Aug 24 10:26:23.791084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 575/1288] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:23 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:23.791240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:23.791240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:23.791240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:23.791546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:23.791891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:23.792384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:23.871063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:23.880090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-77ef6cda-b975-468e-be83-7ce84a5d7364 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.101103 Aug 24 10:26:23.880673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 714/1289] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:23 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:24.741937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1fa519cf-2be2-4c85-9670-486d9bfebb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:24.744165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1fa519cf-2be2-4c85-9670-486d9bfebb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:24.750170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa519cf-2be2-4c85-9670-486d9bfebb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:24.751260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa519cf-2be2-4c85-9670-486d9bfebb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:24.751602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa519cf-2be2-4c85-9670-486d9bfebb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:24.752320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa519cf-2be2-4c85-9670-486d9bfebb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:24.752777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa519cf-2be2-4c85-9670-486d9bfebb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:24.753207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fa519cf-2be2-4c85-9670-486d9bfebb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:24.784529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1fa519cf-2be2-4c85-9670-486d9bfebb50 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046486 Aug 24 10:26:24.785111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 576/1290] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:24 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:24.799299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3f8682f3-8c04-4c44-8d15-017efe2c71f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:24.801469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3f8682f3-8c04-4c44-8d15-017efe2c71f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:24.807865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f8682f3-8c04-4c44-8d15-017efe2c71f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:24.808385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f8682f3-8c04-4c44-8d15-017efe2c71f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:24.808831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f8682f3-8c04-4c44-8d15-017efe2c71f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:24.809525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f8682f3-8c04-4c44-8d15-017efe2c71f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:24.809993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f8682f3-8c04-4c44-8d15-017efe2c71f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:24.810428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f8682f3-8c04-4c44-8d15-017efe2c71f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:24.840738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3f8682f3-8c04-4c44-8d15-017efe2c71f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045816 Aug 24 10:26:24.841446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 715/1291] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:24 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:24.900137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:24.900747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:24.907735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:24.908249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:24.908673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:24.909356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:24.909804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:24.910219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:24.971432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:24.979304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-acbf82a1-2c92-4046-82af-eaab959c8575 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083064 Aug 24 10:26:24.979895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 577/1292] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:24 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:25.796162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b2262600-7b80-491f-98a4-f2c0c00b12f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:25.798307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b2262600-7b80-491f-98a4-f2c0c00b12f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:25.804300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2262600-7b80-491f-98a4-f2c0c00b12f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:25.804872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2262600-7b80-491f-98a4-f2c0c00b12f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:25.805226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2262600-7b80-491f-98a4-f2c0c00b12f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:25.805927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2262600-7b80-491f-98a4-f2c0c00b12f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:25.806384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2262600-7b80-491f-98a4-f2c0c00b12f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:25.806819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2262600-7b80-491f-98a4-f2c0c00b12f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:25.838201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b2262600-7b80-491f-98a4-f2c0c00b12f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045975 Aug 24 10:26:25.838758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 716/1293] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:25 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:25.852667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c2a0a29d-06b6-4866-a5f6-32c619fe6486 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:25.854853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c2a0a29d-06b6-4866-a5f6-32c619fe6486 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:25.861510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2a0a29d-06b6-4866-a5f6-32c619fe6486 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:25.861996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2a0a29d-06b6-4866-a5f6-32c619fe6486 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:25.862434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2a0a29d-06b6-4866-a5f6-32c619fe6486 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:25.863139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2a0a29d-06b6-4866-a5f6-32c619fe6486 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:25.863598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2a0a29d-06b6-4866-a5f6-32c619fe6486 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:25.864025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2a0a29d-06b6-4866-a5f6-32c619fe6486 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:25.867628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:25.893901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c2a0a29d-06b6-4866-a5f6-32c619fe6486 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045384 Aug 24 10:26:25.894458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 578/1294] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:25 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:26.002082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:26.002704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:26.008589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:26.009082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:26.009531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:26.010233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:26.010693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:26.011127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:26.073398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:26.081440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3f88f114-0ed3-4f61-be19-cbefba2aca0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083707 Aug 24 10:26:26.081996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 717/1295] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:25 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:26.850483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0777f1ca-52bc-4bc0-950b-34c551d092b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:26.852631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0777f1ca-52bc-4bc0-950b-34c551d092b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:26.859150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0777f1ca-52bc-4bc0-950b-34c551d092b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:26.860297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0777f1ca-52bc-4bc0-950b-34c551d092b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:26.860579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0777f1ca-52bc-4bc0-950b-34c551d092b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:26.861272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0777f1ca-52bc-4bc0-950b-34c551d092b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:26.861724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0777f1ca-52bc-4bc0-950b-34c551d092b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:26.862148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0777f1ca-52bc-4bc0-950b-34c551d092b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:26.891904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0777f1ca-52bc-4bc0-950b-34c551d092b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045367 Aug 24 10:26:26.892485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 579/1296] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:26 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:26.905960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-67667478-be15-4ca4-9281-e1bf518246d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:26.908091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-67667478-be15-4ca4-9281-e1bf518246d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:26.914407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67667478-be15-4ca4-9281-e1bf518246d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:26.914887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67667478-be15-4ca4-9281-e1bf518246d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:26.915327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67667478-be15-4ca4-9281-e1bf518246d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:26.916007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67667478-be15-4ca4-9281-e1bf518246d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:26.916489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67667478-be15-4ca4-9281-e1bf518246d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:26.916914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67667478-be15-4ca4-9281-e1bf518246d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:26.946875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-67667478-be15-4ca4-9281-e1bf518246d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044801 Aug 24 10:26:26.947429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 718/1297] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:26 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:27.102036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:27.102572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:27.108495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:27.108977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:27.109905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:27.110590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:27.111046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:27.111467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:27.175435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:27.183155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ce4d5fb2-d2de-468a-a4f6-f5f89d495593 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085113 Aug 24 10:26:27.183714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 580/1298] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:27 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:27.904294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7e08dca0-d765-40c0-9122-02be272cd586 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:27.906829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7e08dca0-d765-40c0-9122-02be272cd586 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:27.913101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e08dca0-d765-40c0-9122-02be272cd586 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:27.913584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e08dca0-d765-40c0-9122-02be272cd586 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:27.914015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e08dca0-d765-40c0-9122-02be272cd586 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:27.914706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e08dca0-d765-40c0-9122-02be272cd586 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:27.915155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e08dca0-d765-40c0-9122-02be272cd586 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:27.915583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e08dca0-d765-40c0-9122-02be272cd586 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:27.955545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7e08dca0-d765-40c0-9122-02be272cd586 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055197 Aug 24 10:26:27.956419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 719/1299] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:27 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:27.960819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c24743c4-ccb1-4338-a96b-592734932862 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:27.962970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c24743c4-ccb1-4338-a96b-592734932862 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:27.970102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c24743c4-ccb1-4338-a96b-592734932862 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:27.970593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c24743c4-ccb1-4338-a96b-592734932862 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:27.971027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c24743c4-ccb1-4338-a96b-592734932862 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:27.971734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c24743c4-ccb1-4338-a96b-592734932862 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:27.972229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c24743c4-ccb1-4338-a96b-592734932862 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:27.972655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c24743c4-ccb1-4338-a96b-592734932862 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:28.003714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c24743c4-ccb1-4338-a96b-592734932862 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048334 Aug 24 10:26:28.004317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 581/1300] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:27 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:28.202716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:28.203590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:28.209423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:28.209939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:28.210381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:28.211071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:28.211529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:28.211958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:28.273148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:28.280840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-10aed8ef-c5a2-47a7-bbf8-d562008f1071 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082008 Aug 24 10:26:28.282016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 720/1301] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:28 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:28.967748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-27cdf9e5-50da-4b64-88c1-d99cfbf9be60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:28.970411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-27cdf9e5-50da-4b64-88c1-d99cfbf9be60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:28.976837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27cdf9e5-50da-4b64-88c1-d99cfbf9be60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:28.977316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27cdf9e5-50da-4b64-88c1-d99cfbf9be60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:28.977753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27cdf9e5-50da-4b64-88c1-d99cfbf9be60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:28.978445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27cdf9e5-50da-4b64-88c1-d99cfbf9be60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:28.978897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27cdf9e5-50da-4b64-88c1-d99cfbf9be60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:28.979317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27cdf9e5-50da-4b64-88c1-d99cfbf9be60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:29.008413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-27cdf9e5-50da-4b64-88c1-d99cfbf9be60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044681 Aug 24 10:26:29.008963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 582/1302] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:28 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:29.016850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9e6d2b62-88a0-4558-9203-783a72859071 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:29.019036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9e6d2b62-88a0-4558-9203-783a72859071 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:29.025534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e6d2b62-88a0-4558-9203-783a72859071 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:29.025957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e6d2b62-88a0-4558-9203-783a72859071 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:29.026393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e6d2b62-88a0-4558-9203-783a72859071 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:29.027087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e6d2b62-88a0-4558-9203-783a72859071 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:29.027540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e6d2b62-88a0-4558-9203-783a72859071 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:29.027966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e6d2b62-88a0-4558-9203-783a72859071 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:29.057588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9e6d2b62-88a0-4558-9203-783a72859071 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045382 Aug 24 10:26:29.058143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 721/1303] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:29 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:29.303566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:29.304111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:29.311853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:29.312398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:29.312830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:29.313516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:29.313966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:29.314384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:29.375766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:29.383417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c1440611-d7f7-42a0-a787-dbca7c4bd6d6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085091 Aug 24 10:26:29.383975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 583/1304] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:29 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:30.020584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a70ef239-5b22-420f-9e95-730a63903c86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:30.022811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a70ef239-5b22-420f-9e95-730a63903c86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:30.029454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70ef239-5b22-420f-9e95-730a63903c86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:30.029943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70ef239-5b22-420f-9e95-730a63903c86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:30.030382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70ef239-5b22-420f-9e95-730a63903c86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:30.031717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70ef239-5b22-420f-9e95-730a63903c86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:30.032203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70ef239-5b22-420f-9e95-730a63903c86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:30.032635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70ef239-5b22-420f-9e95-730a63903c86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:30.061522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a70ef239-5b22-420f-9e95-730a63903c86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044966 Aug 24 10:26:30.062086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 722/1305] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:30 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:30.069599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-93355dfb-0d72-4e68-93e7-4842fe3fa287 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:30.071724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-93355dfb-0d72-4e68-93e7-4842fe3fa287 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:30.079040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93355dfb-0d72-4e68-93e7-4842fe3fa287 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:30.079513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93355dfb-0d72-4e68-93e7-4842fe3fa287 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:30.079949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93355dfb-0d72-4e68-93e7-4842fe3fa287 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:30.080665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93355dfb-0d72-4e68-93e7-4842fe3fa287 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:30.081114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93355dfb-0d72-4e68-93e7-4842fe3fa287 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:30.081530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-93355dfb-0d72-4e68-93e7-4842fe3fa287 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:30.111889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-93355dfb-0d72-4e68-93e7-4842fe3fa287 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046106 Aug 24 10:26:30.112520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 584/1306] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:30 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:30.404499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:30.405028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:30.411350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:30.411840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:30.412305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:30.412992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:30.413443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:30.413865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:30.417773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:30.474609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:30.482490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cd669337-2a42-4d9f-8f08-b6f7faf7b62f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081971 Aug 24 10:26:30.483042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 723/1307] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:30 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:31.074071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f9dbbce1-e6aa-45e8-95a2-c4c04cbf28d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:31.076188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f9dbbce1-e6aa-45e8-95a2-c4c04cbf28d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:31.083441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9dbbce1-e6aa-45e8-95a2-c4c04cbf28d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:31.083920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9dbbce1-e6aa-45e8-95a2-c4c04cbf28d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:31.084388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9dbbce1-e6aa-45e8-95a2-c4c04cbf28d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:31.085080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9dbbce1-e6aa-45e8-95a2-c4c04cbf28d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:31.085530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9dbbce1-e6aa-45e8-95a2-c4c04cbf28d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:31.085954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9dbbce1-e6aa-45e8-95a2-c4c04cbf28d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:31.114657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f9dbbce1-e6aa-45e8-95a2-c4c04cbf28d9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044644 Aug 24 10:26:31.115209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 585/1308] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:31 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:31.123918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-edd56d99-07fc-428b-aef8-062e2aded3ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:31.126808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-edd56d99-07fc-428b-aef8-062e2aded3ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:31.133299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edd56d99-07fc-428b-aef8-062e2aded3ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:31.133782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edd56d99-07fc-428b-aef8-062e2aded3ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:31.134354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edd56d99-07fc-428b-aef8-062e2aded3ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:31.135051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edd56d99-07fc-428b-aef8-062e2aded3ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:31.135535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edd56d99-07fc-428b-aef8-062e2aded3ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:31.135959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-edd56d99-07fc-428b-aef8-062e2aded3ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:31.165572 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-edd56d99-07fc-428b-aef8-062e2aded3ac tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045488 Aug 24 10:26:31.166127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 724/1309] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:31 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:31.503510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:31.504145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:31.510161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:31.510630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:31.511053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:31.511724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:31.512199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:31.512616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:31.572743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:31.580995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c7ed6ec1-6d09-4e86-8637-a9662794b140 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081471 Aug 24 10:26:31.581543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 586/1310] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:31 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:32.127588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8131bce2-1cca-4fbe-b1cf-4efd0ba03fa5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:32.129639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8131bce2-1cca-4fbe-b1cf-4efd0ba03fa5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:32.135912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8131bce2-1cca-4fbe-b1cf-4efd0ba03fa5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:32.136415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8131bce2-1cca-4fbe-b1cf-4efd0ba03fa5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:32.136845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8131bce2-1cca-4fbe-b1cf-4efd0ba03fa5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:32.137532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8131bce2-1cca-4fbe-b1cf-4efd0ba03fa5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:32.137982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8131bce2-1cca-4fbe-b1cf-4efd0ba03fa5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:32.138403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8131bce2-1cca-4fbe-b1cf-4efd0ba03fa5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:32.166829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8131bce2-1cca-4fbe-b1cf-4efd0ba03fa5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043911 Aug 24 10:26:32.167366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 725/1311] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:32 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:32.178204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3f190e26-7854-43c8-a8fc-1d23ed203873 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:32.180390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3f190e26-7854-43c8-a8fc-1d23ed203873 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:32.186774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f190e26-7854-43c8-a8fc-1d23ed203873 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:32.187895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f190e26-7854-43c8-a8fc-1d23ed203873 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:32.188360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f190e26-7854-43c8-a8fc-1d23ed203873 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:32.189034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f190e26-7854-43c8-a8fc-1d23ed203873 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:32.189480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f190e26-7854-43c8-a8fc-1d23ed203873 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:32.189894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f190e26-7854-43c8-a8fc-1d23ed203873 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:32.218407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3f190e26-7854-43c8-a8fc-1d23ed203873 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044133 Aug 24 10:26:32.218951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 587/1312] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:32 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:32.601672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:32.602279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:32.608182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:32.608662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:32.609719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:32.610396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:32.610847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:32.611268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:32.675113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:32.682834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cd5c610c-3e56-41d7-8be3-5564901d2e08 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085005 Aug 24 10:26:32.683389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 726/1313] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:32 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:33.178999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-feac695d-afab-4563-8b04-cf1ef7635d7f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:33.181157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-feac695d-afab-4563-8b04-cf1ef7635d7f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:33.187989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-feac695d-afab-4563-8b04-cf1ef7635d7f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:33.188502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-feac695d-afab-4563-8b04-cf1ef7635d7f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:33.188958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-feac695d-afab-4563-8b04-cf1ef7635d7f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:33.189644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-feac695d-afab-4563-8b04-cf1ef7635d7f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:33.190097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-feac695d-afab-4563-8b04-cf1ef7635d7f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:33.190521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-feac695d-afab-4563-8b04-cf1ef7635d7f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:33.220292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-feac695d-afab-4563-8b04-cf1ef7635d7f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045176 Aug 24 10:26:33.220858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 588/1314] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:33 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:33.230707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-10f542fd-8335-498d-9151-3d16643d658f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:33.233151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-10f542fd-8335-498d-9151-3d16643d658f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:33.239494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10f542fd-8335-498d-9151-3d16643d658f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:33.239978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10f542fd-8335-498d-9151-3d16643d658f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:33.240450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10f542fd-8335-498d-9151-3d16643d658f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:33.241143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10f542fd-8335-498d-9151-3d16643d658f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:33.241599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10f542fd-8335-498d-9151-3d16643d658f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:33.242026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10f542fd-8335-498d-9151-3d16643d658f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:33.271507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-10f542fd-8335-498d-9151-3d16643d658f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044861 Aug 24 10:26:33.272068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 727/1315] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:33 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:33.703647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:33.704326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:33.710232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:33.710715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:33.711150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:33.711842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:33.712318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:33.712747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:33.773749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:33.782875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d5ba9d70-57e4-4db1-88af-400fae2ee1df tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083720 Aug 24 10:26:33.783443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 589/1316] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:33 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:34.232678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5ca7a5bf-3456-4856-8185-43f65da1781d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:34.235362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5ca7a5bf-3456-4856-8185-43f65da1781d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:34.241737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca7a5bf-3456-4856-8185-43f65da1781d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:34.242220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca7a5bf-3456-4856-8185-43f65da1781d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:34.242658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca7a5bf-3456-4856-8185-43f65da1781d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:34.243350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca7a5bf-3456-4856-8185-43f65da1781d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:34.243802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca7a5bf-3456-4856-8185-43f65da1781d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:34.244261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca7a5bf-3456-4856-8185-43f65da1781d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:34.273797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5ca7a5bf-3456-4856-8185-43f65da1781d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045107 Aug 24 10:26:34.274344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 728/1317] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:34 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:34.283574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-47d4864e-75f4-4643-83d7-71da72bc250e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:34.285904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-47d4864e-75f4-4643-83d7-71da72bc250e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:34.292295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47d4864e-75f4-4643-83d7-71da72bc250e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:34.292775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47d4864e-75f4-4643-83d7-71da72bc250e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:34.293202 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47d4864e-75f4-4643-83d7-71da72bc250e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:34.293881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47d4864e-75f4-4643-83d7-71da72bc250e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:34.294329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47d4864e-75f4-4643-83d7-71da72bc250e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:34.294750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47d4864e-75f4-4643-83d7-71da72bc250e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:34.323402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-47d4864e-75f4-4643-83d7-71da72bc250e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044416 Aug 24 10:26:34.323959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 590/1318] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:34 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:34.802752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:34.803372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:34.810115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:34.811412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:34.812564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:34.813283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:34.813747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:34.814177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:34.876152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:34.883834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2358cb03-e70b-47fc-8501-7a0beb183cca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084921 Aug 24 10:26:34.884417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 729/1319] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:34 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:35.286539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b77a3979-8457-41bc-9772-f681b34289bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:35.288741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b77a3979-8457-41bc-9772-f681b34289bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:35.295176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77a3979-8457-41bc-9772-f681b34289bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:35.295646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77a3979-8457-41bc-9772-f681b34289bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:35.296098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77a3979-8457-41bc-9772-f681b34289bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:35.297284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77a3979-8457-41bc-9772-f681b34289bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:35.297728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77a3979-8457-41bc-9772-f681b34289bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:35.298147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77a3979-8457-41bc-9772-f681b34289bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:35.327571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b77a3979-8457-41bc-9772-f681b34289bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044900 Aug 24 10:26:35.328399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 591/1320] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:35 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:35.335958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c7fad489-0e88-4348-8316-95c2567b538d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:35.338150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c7fad489-0e88-4348-8316-95c2567b538d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:35.344592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7fad489-0e88-4348-8316-95c2567b538d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:35.345071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7fad489-0e88-4348-8316-95c2567b538d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:35.345504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7fad489-0e88-4348-8316-95c2567b538d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:35.346188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7fad489-0e88-4348-8316-95c2567b538d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:35.346638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7fad489-0e88-4348-8316-95c2567b538d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:35.347064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7fad489-0e88-4348-8316-95c2567b538d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:35.376473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c7fad489-0e88-4348-8316-95c2567b538d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044437 Aug 24 10:26:35.377012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 730/1321] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:35 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:35.904275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:35.904886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:35.911382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:35.911860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:35.912319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:35.913103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:35.913564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:35.913984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:35.977665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:35.985311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-479aba8e-1874-44b9-9108-bc14e4ffe039 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085072 Aug 24 10:26:35.985856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 592/1322] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:35 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:36.339839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cc34b3b9-4fd5-4b73-a4fd-7993d8815fa8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:36.342052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cc34b3b9-4fd5-4b73-a4fd-7993d8815fa8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:36.349158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc34b3b9-4fd5-4b73-a4fd-7993d8815fa8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:36.349639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc34b3b9-4fd5-4b73-a4fd-7993d8815fa8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:36.350070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc34b3b9-4fd5-4b73-a4fd-7993d8815fa8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:36.350763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc34b3b9-4fd5-4b73-a4fd-7993d8815fa8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:36.351212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc34b3b9-4fd5-4b73-a4fd-7993d8815fa8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:36.351627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc34b3b9-4fd5-4b73-a4fd-7993d8815fa8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:36.381143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cc34b3b9-4fd5-4b73-a4fd-7993d8815fa8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045240 Aug 24 10:26:36.381693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 731/1323] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:36 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:36.388903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3a49b287-7f41-4906-91f0-b333ea62dbfb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:36.391084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3a49b287-7f41-4906-91f0-b333ea62dbfb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:36.397441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a49b287-7f41-4906-91f0-b333ea62dbfb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:36.397918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a49b287-7f41-4906-91f0-b333ea62dbfb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:36.398348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a49b287-7f41-4906-91f0-b333ea62dbfb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:36.399032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a49b287-7f41-4906-91f0-b333ea62dbfb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:36.399510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a49b287-7f41-4906-91f0-b333ea62dbfb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:36.399932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a49b287-7f41-4906-91f0-b333ea62dbfb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:36.430039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3a49b287-7f41-4906-91f0-b333ea62dbfb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045284 Aug 24 10:26:36.430585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 593/1324] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:36 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:37.006346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:37.006950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:37.012916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:37.013393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:37.013830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:37.014521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:37.015547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:37.015971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:37.076457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:37.084638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4c0f3468-d069-43f8-8383-49685ec1a141 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082182 Aug 24 10:26:37.085182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 732/1325] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:37 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:37.394696 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5136c8a2-c7c9-49f0-9e90-8a7dc5b085d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:37.396761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5136c8a2-c7c9-49f0-9e90-8a7dc5b085d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:37.403155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5136c8a2-c7c9-49f0-9e90-8a7dc5b085d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:37.403656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5136c8a2-c7c9-49f0-9e90-8a7dc5b085d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:37.404107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5136c8a2-c7c9-49f0-9e90-8a7dc5b085d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:37.404820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5136c8a2-c7c9-49f0-9e90-8a7dc5b085d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:37.405261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5136c8a2-c7c9-49f0-9e90-8a7dc5b085d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:37.406280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5136c8a2-c7c9-49f0-9e90-8a7dc5b085d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:37.430308 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:37.435461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5136c8a2-c7c9-49f0-9e90-8a7dc5b085d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045408 Aug 24 10:26:37.436008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 594/1326] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:37 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:37.441863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-51cad9f7-876a-40c6-ac0f-f3d1125b37ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:37.443967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-51cad9f7-876a-40c6-ac0f-f3d1125b37ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:37.450423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51cad9f7-876a-40c6-ac0f-f3d1125b37ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:37.450726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51cad9f7-876a-40c6-ac0f-f3d1125b37ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:37.451164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51cad9f7-876a-40c6-ac0f-f3d1125b37ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:37.451841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51cad9f7-876a-40c6-ac0f-f3d1125b37ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:37.453084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51cad9f7-876a-40c6-ac0f-f3d1125b37ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:37.453504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51cad9f7-876a-40c6-ac0f-f3d1125b37ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:37.482659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-51cad9f7-876a-40c6-ac0f-f3d1125b37ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044671 Aug 24 10:26:37.483222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 733/1327] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:37 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:38.105545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:38.106143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:38.112675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:38.113159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:38.113593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:38.114280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:38.114730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:38.115149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:38.178835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:38.186667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-458fff47-5c41-49cc-8739-ae346812ca27 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085296 Aug 24 10:26:38.187761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 595/1328] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:38 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:38.447747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c4fe8a48-9079-4422-b37c-c630ed70d8e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:38.449913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c4fe8a48-9079-4422-b37c-c630ed70d8e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:38.456563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4fe8a48-9079-4422-b37c-c630ed70d8e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:38.457046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4fe8a48-9079-4422-b37c-c630ed70d8e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:38.457485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4fe8a48-9079-4422-b37c-c630ed70d8e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:38.458176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4fe8a48-9079-4422-b37c-c630ed70d8e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:38.458629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4fe8a48-9079-4422-b37c-c630ed70d8e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:38.459053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4fe8a48-9079-4422-b37c-c630ed70d8e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:38.489041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c4fe8a48-9079-4422-b37c-c630ed70d8e6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045292 Aug 24 10:26:38.489694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 734/1329] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:38 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:38.494909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1c364646-c2bf-4f24-a906-83706ee5d20e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:38.497142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1c364646-c2bf-4f24-a906-83706ee5d20e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:38.503703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c364646-c2bf-4f24-a906-83706ee5d20e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:38.504219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c364646-c2bf-4f24-a906-83706ee5d20e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:38.504657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c364646-c2bf-4f24-a906-83706ee5d20e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:38.505345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c364646-c2bf-4f24-a906-83706ee5d20e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:38.505798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c364646-c2bf-4f24-a906-83706ee5d20e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:38.506223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c364646-c2bf-4f24-a906-83706ee5d20e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:38.535075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1c364646-c2bf-4f24-a906-83706ee5d20e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044074 Aug 24 10:26:38.535640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 596/1330] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:38 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:39.207059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:39.207671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:39.213730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:39.214206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:39.214640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:39.215319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:39.215769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:39.216228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:39.276695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:39.287240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-72e5c1c0-16c4-4596-8423-9c0f0a723e2d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083904 Aug 24 10:26:39.287817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 735/1331] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:39 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:39.502678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7e299415-8457-4fda-837b-4360dd8881a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:39.504794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7e299415-8457-4fda-837b-4360dd8881a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:39.511199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e299415-8457-4fda-837b-4360dd8881a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:39.511676 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e299415-8457-4fda-837b-4360dd8881a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:39.512143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e299415-8457-4fda-837b-4360dd8881a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:39.513011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e299415-8457-4fda-837b-4360dd8881a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:39.513267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e299415-8457-4fda-837b-4360dd8881a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:39.513690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e299415-8457-4fda-837b-4360dd8881a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:39.542799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7e299415-8457-4fda-837b-4360dd8881a6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044827 Aug 24 10:26:39.543362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 597/1332] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:39 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:39.547519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-60e59d4f-866b-440f-af67-1d7cc73f98eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:39.549679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-60e59d4f-866b-440f-af67-1d7cc73f98eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:39.556147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60e59d4f-866b-440f-af67-1d7cc73f98eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:39.556646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60e59d4f-866b-440f-af67-1d7cc73f98eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:39.557086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60e59d4f-866b-440f-af67-1d7cc73f98eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:39.557780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60e59d4f-866b-440f-af67-1d7cc73f98eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:39.558270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60e59d4f-866b-440f-af67-1d7cc73f98eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:39.558700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60e59d4f-866b-440f-af67-1d7cc73f98eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:39.588114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-60e59d4f-866b-440f-af67-1d7cc73f98eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045160 Aug 24 10:26:39.588664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 736/1333] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:39 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:40.308302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:40.308919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:40.315350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:40.315831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:40.316302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:40.317012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:40.317461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:40.317889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:40.379551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:40.387257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e479ffa0-2adf-4d43-b694-ec229885dde9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082866 Aug 24 10:26:40.387815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 598/1334] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:40 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:40.555967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dc8f2489-51cc-49e1-93d9-8669b1e2b0a4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:40.558192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dc8f2489-51cc-49e1-93d9-8669b1e2b0a4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:40.565134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8f2489-51cc-49e1-93d9-8669b1e2b0a4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:40.565615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8f2489-51cc-49e1-93d9-8669b1e2b0a4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:40.566054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8f2489-51cc-49e1-93d9-8669b1e2b0a4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:40.566744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8f2489-51cc-49e1-93d9-8669b1e2b0a4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:40.567218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8f2489-51cc-49e1-93d9-8669b1e2b0a4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:40.567628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc8f2489-51cc-49e1-93d9-8669b1e2b0a4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:40.596287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dc8f2489-51cc-49e1-93d9-8669b1e2b0a4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044293 Aug 24 10:26:40.596833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 737/1335] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:40 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:40.601270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c807bdc0-f283-49da-8c98-d433ca7c9eb9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:40.603388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c807bdc0-f283-49da-8c98-d433ca7c9eb9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:40.610885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c807bdc0-f283-49da-8c98-d433ca7c9eb9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:40.611289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c807bdc0-f283-49da-8c98-d433ca7c9eb9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:40.611723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c807bdc0-f283-49da-8c98-d433ca7c9eb9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:40.612427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c807bdc0-f283-49da-8c98-d433ca7c9eb9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:40.612878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c807bdc0-f283-49da-8c98-d433ca7c9eb9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:40.613303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c807bdc0-f283-49da-8c98-d433ca7c9eb9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:40.641959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c807bdc0-f283-49da-8c98-d433ca7c9eb9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044898 Aug 24 10:26:40.642506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 599/1336] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:40 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:41.407371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:41.407987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:41.413983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:41.414473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:41.414916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:41.415604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:41.416081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:41.416513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:41.480378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:41.488196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1c04bc50-811f-413b-8715-d6864a2d08a4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085255 Aug 24 10:26:41.488761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 738/1337] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:41 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:41.608432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bfdd7355-1d29-42c7-9dda-d3f088e0b177 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:41.610598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bfdd7355-1d29-42c7-9dda-d3f088e0b177 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:41.616958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfdd7355-1d29-42c7-9dda-d3f088e0b177 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:41.617429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfdd7355-1d29-42c7-9dda-d3f088e0b177 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:41.617854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfdd7355-1d29-42c7-9dda-d3f088e0b177 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:41.618530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfdd7355-1d29-42c7-9dda-d3f088e0b177 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:41.618972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfdd7355-1d29-42c7-9dda-d3f088e0b177 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:41.619387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfdd7355-1d29-42c7-9dda-d3f088e0b177 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:41.650080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bfdd7355-1d29-42c7-9dda-d3f088e0b177 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045569 Aug 24 10:26:41.650632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 600/1338] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:41 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:41.654223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ba4a60ce-f451-44ca-a0b8-756f7ca5ebae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:41.656427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ba4a60ce-f451-44ca-a0b8-756f7ca5ebae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:41.662718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba4a60ce-f451-44ca-a0b8-756f7ca5ebae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:41.663202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba4a60ce-f451-44ca-a0b8-756f7ca5ebae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:41.663641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba4a60ce-f451-44ca-a0b8-756f7ca5ebae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:41.664356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba4a60ce-f451-44ca-a0b8-756f7ca5ebae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:41.664811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba4a60ce-f451-44ca-a0b8-756f7ca5ebae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:41.665239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba4a60ce-f451-44ca-a0b8-756f7ca5ebae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:41.689916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:41.694966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ba4a60ce-f451-44ca-a0b8-756f7ca5ebae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044816 Aug 24 10:26:41.695507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 739/1339] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:41 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:42.508702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:42.509313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:42.515746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:42.516282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:42.516711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:42.517393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:42.517843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:42.518262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:42.578503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:42.586163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-20ba0321-f3f3-40c6-a554-24206efb2d4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081335 Aug 24 10:26:42.586720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 601/1340] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:42 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:42.662765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-79c83542-0f94-4377-937a-acb9d1861de1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:42.664950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-79c83542-0f94-4377-937a-acb9d1861de1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:42.671200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79c83542-0f94-4377-937a-acb9d1861de1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:42.671679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79c83542-0f94-4377-937a-acb9d1861de1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:42.672151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79c83542-0f94-4377-937a-acb9d1861de1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:42.672835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79c83542-0f94-4377-937a-acb9d1861de1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:42.673289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79c83542-0f94-4377-937a-acb9d1861de1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:42.673713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79c83542-0f94-4377-937a-acb9d1861de1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:42.703128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-79c83542-0f94-4377-937a-acb9d1861de1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044560 Aug 24 10:26:42.703819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 740/1341] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:42 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:42.707441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cb421dbd-3744-46c0-99b6-15126b37212a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:42.709601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cb421dbd-3744-46c0-99b6-15126b37212a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:42.716018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb421dbd-3744-46c0-99b6-15126b37212a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:42.716527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb421dbd-3744-46c0-99b6-15126b37212a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:42.716961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb421dbd-3744-46c0-99b6-15126b37212a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:42.718379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb421dbd-3744-46c0-99b6-15126b37212a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:42.718841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb421dbd-3744-46c0-99b6-15126b37212a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:42.719282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb421dbd-3744-46c0-99b6-15126b37212a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:42.747479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cb421dbd-3744-46c0-99b6-15126b37212a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043936 Aug 24 10:26:42.748023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 602/1342] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:42 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:43.607459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:43.607992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:43.614523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:43.615034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:43.615481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:43.616200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:43.616658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:43.617108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:43.678273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:43.685808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-436dff84-8e1b-4c39-89d5-ffd47d73e0f7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082412 Aug 24 10:26:43.686849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 741/1343] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:43 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:43.714801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bc74ac3e-8bf4-4ac4-bebb-4c38062bb274 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:43.716989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bc74ac3e-8bf4-4ac4-bebb-4c38062bb274 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:43.723404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc74ac3e-8bf4-4ac4-bebb-4c38062bb274 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:43.723887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc74ac3e-8bf4-4ac4-bebb-4c38062bb274 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:43.724385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc74ac3e-8bf4-4ac4-bebb-4c38062bb274 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:43.725077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc74ac3e-8bf4-4ac4-bebb-4c38062bb274 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:43.725532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc74ac3e-8bf4-4ac4-bebb-4c38062bb274 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:43.725959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc74ac3e-8bf4-4ac4-bebb-4c38062bb274 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:43.755252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bc74ac3e-8bf4-4ac4-bebb-4c38062bb274 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044341 Aug 24 10:26:43.755818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 603/1344] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:43 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:43.760512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9e9783e4-6168-4a71-a2f0-254c78d67f37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:43.762637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9e9783e4-6168-4a71-a2f0-254c78d67f37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:43.769332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e9783e4-6168-4a71-a2f0-254c78d67f37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:43.769817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e9783e4-6168-4a71-a2f0-254c78d67f37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:43.770254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e9783e4-6168-4a71-a2f0-254c78d67f37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:43.770949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e9783e4-6168-4a71-a2f0-254c78d67f37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:43.771609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e9783e4-6168-4a71-a2f0-254c78d67f37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:43.771806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e9783e4-6168-4a71-a2f0-254c78d67f37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:43.800542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9e9783e4-6168-4a71-a2f0-254c78d67f37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044959 Aug 24 10:26:43.801081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 742/1345] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:43 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:44.258672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:26:44.258925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:26:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:26:44.707155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:44.707772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:44.713928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:44.714409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:44.714841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:44.715530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:44.716005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:44.716457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:44.767874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c4d4e8dc-4e1e-4f8f-a694-e15853d48db8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:44.770243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c4d4e8dc-4e1e-4f8f-a694-e15853d48db8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:44.777414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e8dc-4e1e-4f8f-a694-e15853d48db8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:44.777414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e8dc-4e1e-4f8f-a694-e15853d48db8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:44.777880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e8dc-4e1e-4f8f-a694-e15853d48db8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:44.778811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e8dc-4e1e-4f8f-a694-e15853d48db8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:44.779480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e8dc-4e1e-4f8f-a694-e15853d48db8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:44.779942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4d4e8dc-4e1e-4f8f-a694-e15853d48db8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:44.784255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:44.793541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-31c49cff-5cd6-415a-a324-64ece94e287f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.090190 Aug 24 10:26:44.794777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 604/1346] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:44 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:44.815100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-16ddbbbf-4eb6-4db1-8eae-b6e126b2ced2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:44.817322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-16ddbbbf-4eb6-4db1-8eae-b6e126b2ced2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:44.823854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c4d4e8dc-4e1e-4f8f-a694-e15853d48db8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060522 Aug 24 10:26:44.824442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 743/1347] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:44 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:44.825669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ddbbbf-4eb6-4db1-8eae-b6e126b2ced2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:44.826159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ddbbbf-4eb6-4db1-8eae-b6e126b2ced2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:44.826595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ddbbbf-4eb6-4db1-8eae-b6e126b2ced2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:44.828103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ddbbbf-4eb6-4db1-8eae-b6e126b2ced2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:44.828581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ddbbbf-4eb6-4db1-8eae-b6e126b2ced2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:44.828989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ddbbbf-4eb6-4db1-8eae-b6e126b2ced2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:44.863583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-16ddbbbf-4eb6-4db1-8eae-b6e126b2ced2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054011 Aug 24 10:26:44.864156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 605/1348] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:44 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:45.815151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:45.815769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:45.822163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:45.822648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:45.823086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:45.823770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:45.824255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:45.824681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:45.836923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1e6e6f08-ddd2-42d7-91c8-0cab43c3192b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:45.839047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1e6e6f08-ddd2-42d7-91c8-0cab43c3192b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:45.846288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6e6f08-ddd2-42d7-91c8-0cab43c3192b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:45.846768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6e6f08-ddd2-42d7-91c8-0cab43c3192b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:45.847199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6e6f08-ddd2-42d7-91c8-0cab43c3192b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:45.847885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6e6f08-ddd2-42d7-91c8-0cab43c3192b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:45.848361 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6e6f08-ddd2-42d7-91c8-0cab43c3192b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:45.848778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e6e6f08-ddd2-42d7-91c8-0cab43c3192b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:45.879924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1e6e6f08-ddd2-42d7-91c8-0cab43c3192b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046801 Aug 24 10:26:45.880516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 606/1349] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:45 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:45.885085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-688e58ed-ee30-4631-8e69-a143b0883d56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:45.886597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:45.887199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-688e58ed-ee30-4631-8e69-a143b0883d56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:45.893048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688e58ed-ee30-4631-8e69-a143b0883d56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:45.893534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688e58ed-ee30-4631-8e69-a143b0883d56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:45.893970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688e58ed-ee30-4631-8e69-a143b0883d56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:45.894545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d833f4ab-6566-4214-9c6e-f955b7f09ea8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083926 Aug 24 10:26:45.894663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688e58ed-ee30-4631-8e69-a143b0883d56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:45.895104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 744/1350] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:45 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:45.895322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688e58ed-ee30-4631-8e69-a143b0883d56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:45.895525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688e58ed-ee30-4631-8e69-a143b0883d56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:45.925071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-688e58ed-ee30-4631-8e69-a143b0883d56 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043556 Aug 24 10:26:45.925616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 607/1351] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:45 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:46.892448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1a7e2910-d324-4af2-b593-374b7feddeff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:46.894579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1a7e2910-d324-4af2-b593-374b7feddeff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:46.901898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1a7e2910-d324-4af2-b593-374b7feddeff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:46.902389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1a7e2910-d324-4af2-b593-374b7feddeff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:46.902830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1a7e2910-d324-4af2-b593-374b7feddeff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:46.903526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1a7e2910-d324-4af2-b593-374b7feddeff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:46.903983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1a7e2910-d324-4af2-b593-374b7feddeff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:46.904440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1a7e2910-d324-4af2-b593-374b7feddeff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:46.914704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:46.915326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:46.921652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:46.922133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:46.922569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:46.923264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:46.923721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:46.924172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:46.933232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1a7e2910-d324-4af2-b593-374b7feddeff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044718 Aug 24 10:26:46.933838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 745/1352] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:46 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:46.939072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-174cb012-e997-4fee-b67d-972453b752d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:46.941204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-174cb012-e997-4fee-b67d-972453b752d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:46.946823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-174cb012-e997-4fee-b67d-972453b752d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:46.947312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-174cb012-e997-4fee-b67d-972453b752d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:46.947751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-174cb012-e997-4fee-b67d-972453b752d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:46.948469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-174cb012-e997-4fee-b67d-972453b752d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:46.948928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-174cb012-e997-4fee-b67d-972453b752d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:46.949352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-174cb012-e997-4fee-b67d-972453b752d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:46.980092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-174cb012-e997-4fee-b67d-972453b752d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045171 Aug 24 10:26:46.980684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 746/1353] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:46 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:47.229092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:47.237263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0e17588a-180b-4734-a7d1-190001cd597e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.326475 Aug 24 10:26:47.237827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 608/1354] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:46 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:47.945948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b3a478e0-e524-4e84-88ae-268461e620b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:47.948259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b3a478e0-e524-4e84-88ae-268461e620b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:47.955360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3a478e0-e524-4e84-88ae-268461e620b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:47.955845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3a478e0-e524-4e84-88ae-268461e620b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:47.956328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3a478e0-e524-4e84-88ae-268461e620b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:47.957025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3a478e0-e524-4e84-88ae-268461e620b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:47.957483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3a478e0-e524-4e84-88ae-268461e620b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:47.957907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3a478e0-e524-4e84-88ae-268461e620b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:47.986737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b3a478e0-e524-4e84-88ae-268461e620b5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044742 Aug 24 10:26:47.987285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 747/1355] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:47 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:47.991476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9385d451-5f1d-4400-85a1-b8730e4d821c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:47.993699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9385d451-5f1d-4400-85a1-b8730e4d821c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:48.000126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9385d451-5f1d-4400-85a1-b8730e4d821c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:48.000602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9385d451-5f1d-4400-85a1-b8730e4d821c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:48.001028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9385d451-5f1d-4400-85a1-b8730e4d821c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:48.001707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9385d451-5f1d-4400-85a1-b8730e4d821c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:48.002154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9385d451-5f1d-4400-85a1-b8730e4d821c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:48.002568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9385d451-5f1d-4400-85a1-b8730e4d821c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:48.032698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9385d451-5f1d-4400-85a1-b8730e4d821c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045032 Aug 24 10:26:48.033172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 609/1356] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:47 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:48.257782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:48.258396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:48.264989 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:48.265477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:48.265913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:48.266598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:48.267053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:48.267477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:48.328606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:48.336151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-82d3e88c-7651-4afb-82e1-0006b6d36194 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082215 Aug 24 10:26:48.336705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 748/1357] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:48 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:48.748103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:26:48.748333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:26:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:26:48.998900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-12f8f379-0f9a-4384-99c9-f2c43e31b724 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:49.001076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-12f8f379-0f9a-4384-99c9-f2c43e31b724 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:49.007629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12f8f379-0f9a-4384-99c9-f2c43e31b724 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:49.008149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12f8f379-0f9a-4384-99c9-f2c43e31b724 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:49.008583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12f8f379-0f9a-4384-99c9-f2c43e31b724 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:49.009301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12f8f379-0f9a-4384-99c9-f2c43e31b724 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:49.009751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12f8f379-0f9a-4384-99c9-f2c43e31b724 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:49.010172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-12f8f379-0f9a-4384-99c9-f2c43e31b724 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:49.039311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-12f8f379-0f9a-4384-99c9-f2c43e31b724 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044348 Aug 24 10:26:49.039866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 610/1358] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:48 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:49.044873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-89243faf-0ba4-412d-aae7-75fc4d290df8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:49.047873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-89243faf-0ba4-412d-aae7-75fc4d290df8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:49.054243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89243faf-0ba4-412d-aae7-75fc4d290df8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:49.054727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89243faf-0ba4-412d-aae7-75fc4d290df8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:49.055167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89243faf-0ba4-412d-aae7-75fc4d290df8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:49.055862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89243faf-0ba4-412d-aae7-75fc4d290df8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:49.056358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89243faf-0ba4-412d-aae7-75fc4d290df8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:49.056787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89243faf-0ba4-412d-aae7-75fc4d290df8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:49.085360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-89243faf-0ba4-412d-aae7-75fc4d290df8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044536 Aug 24 10:26:49.085901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 749/1359] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:49 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:49.356800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:49.357331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:49.363719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:49.364237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:49.364664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:49.365345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:49.365789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:49.366207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:49.384358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:49.427508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:49.435323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4fc55504-48c4-416c-910e-1f857297cb47 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082595 Aug 24 10:26:49.436515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 611/1360] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:49 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:50.051054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c614d67d-2172-4557-bc7f-6f0cab0a9593 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:50.060336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c614d67d-2172-4557-bc7f-6f0cab0a9593 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:50.066712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c614d67d-2172-4557-bc7f-6f0cab0a9593 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:50.067209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c614d67d-2172-4557-bc7f-6f0cab0a9593 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:50.067649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c614d67d-2172-4557-bc7f-6f0cab0a9593 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:50.068410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c614d67d-2172-4557-bc7f-6f0cab0a9593 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:50.068869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c614d67d-2172-4557-bc7f-6f0cab0a9593 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:50.069299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c614d67d-2172-4557-bc7f-6f0cab0a9593 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:50.098324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-683451e6-c83f-4a3b-b077-dbbb22e1c917 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:50.100536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-683451e6-c83f-4a3b-b077-dbbb22e1c917 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:50.102421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=86 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:50.103594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c614d67d-2172-4557-bc7f-6f0cab0a9593 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056545 Aug 24 10:26:50.104430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 750/1361] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:50 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:50.107490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-683451e6-c83f-4a3b-b077-dbbb22e1c917 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:50.108068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-683451e6-c83f-4a3b-b077-dbbb22e1c917 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:50.108587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-683451e6-c83f-4a3b-b077-dbbb22e1c917 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:50.109285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-683451e6-c83f-4a3b-b077-dbbb22e1c917 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:50.109741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-683451e6-c83f-4a3b-b077-dbbb22e1c917 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:50.110167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-683451e6-c83f-4a3b-b077-dbbb22e1c917 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:50.139703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-683451e6-c83f-4a3b-b077-dbbb22e1c917 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044781 Aug 24 10:26:50.140281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 612/1362] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:50 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:50.457628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:50.458247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:50.464566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:50.465048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:50.465484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:50.466169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:50.466624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:50.467179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:50.531443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:50.539147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-24c31e2f-2f16-46d8-b37b-02399f513b8e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085771 Aug 24 10:26:50.539706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 751/1363] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:50 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:51.116247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-94fde75a-e769-4e51-819c-8af6b7b9197a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:51.118292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-94fde75a-e769-4e51-819c-8af6b7b9197a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:51.125599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94fde75a-e769-4e51-819c-8af6b7b9197a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:51.126079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94fde75a-e769-4e51-819c-8af6b7b9197a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:51.126516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94fde75a-e769-4e51-819c-8af6b7b9197a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:51.127216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94fde75a-e769-4e51-819c-8af6b7b9197a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:51.127670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94fde75a-e769-4e51-819c-8af6b7b9197a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:51.128118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94fde75a-e769-4e51-819c-8af6b7b9197a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:51.152163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2d349f4a-7794-48d7-99ba-3cd73351894f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:51.155014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2d349f4a-7794-48d7-99ba-3cd73351894f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:51.157434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=88 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:51.159179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-94fde75a-e769-4e51-819c-8af6b7b9197a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046974 Aug 24 10:26:51.159750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 613/1364] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:51 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:51.162746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d349f4a-7794-48d7-99ba-3cd73351894f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:51.163240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d349f4a-7794-48d7-99ba-3cd73351894f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:51.163681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d349f4a-7794-48d7-99ba-3cd73351894f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:51.164424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d349f4a-7794-48d7-99ba-3cd73351894f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:51.164885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d349f4a-7794-48d7-99ba-3cd73351894f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:51.165316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d349f4a-7794-48d7-99ba-3cd73351894f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:51.195984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2d349f4a-7794-48d7-99ba-3cd73351894f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047886 Aug 24 10:26:51.196575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 752/1365] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:51 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:51.559090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:51.559699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:51.565912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:51.566391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:51.566826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:51.567504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:51.567953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:51.568404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:51.627532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:51.635014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-969f6448-520f-4f9d-afb8-529609a1a406 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.079820 Aug 24 10:26:51.635575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 614/1366] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:51 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:52.171442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ce41763d-1cea-4df0-a4b9-afbaee267f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:52.173608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ce41763d-1cea-4df0-a4b9-afbaee267f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:52.180129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce41763d-1cea-4df0-a4b9-afbaee267f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:52.180609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce41763d-1cea-4df0-a4b9-afbaee267f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:52.181040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce41763d-1cea-4df0-a4b9-afbaee267f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:52.181726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce41763d-1cea-4df0-a4b9-afbaee267f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:52.182176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce41763d-1cea-4df0-a4b9-afbaee267f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:52.182596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce41763d-1cea-4df0-a4b9-afbaee267f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:52.209241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a20d5fa4-dc76-4acb-9f99-c84ab3ea7221 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:52.211402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a20d5fa4-dc76-4acb-9f99-c84ab3ea7221 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:52.212330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ce41763d-1cea-4df0-a4b9-afbaee267f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044712 Aug 24 10:26:52.212886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 753/1367] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:52 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:52.218050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a20d5fa4-dc76-4acb-9f99-c84ab3ea7221 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:52.218535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a20d5fa4-dc76-4acb-9f99-c84ab3ea7221 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:52.218968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a20d5fa4-dc76-4acb-9f99-c84ab3ea7221 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:52.219660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a20d5fa4-dc76-4acb-9f99-c84ab3ea7221 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:52.220136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a20d5fa4-dc76-4acb-9f99-c84ab3ea7221 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:52.220565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a20d5fa4-dc76-4acb-9f99-c84ab3ea7221 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:52.249131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a20d5fa4-dc76-4acb-9f99-c84ab3ea7221 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043823 Aug 24 10:26:52.250183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 615/1368] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:52 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:52.655951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:52.656585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:52.662640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:52.663122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:52.663563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:52.664286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:52.664739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:52.665161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:52.725937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:52.733762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eefd31b9-ad1d-4dc1-a837-7fcdd7d7a4ed tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081691 Aug 24 10:26:52.734350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 754/1369] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:52 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:53.224070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5007e193-cf1a-45d5-89d9-bf95fe9ee04b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:53.226217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5007e193-cf1a-45d5-89d9-bf95fe9ee04b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:53.232663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5007e193-cf1a-45d5-89d9-bf95fe9ee04b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:53.233141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5007e193-cf1a-45d5-89d9-bf95fe9ee04b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:53.233573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5007e193-cf1a-45d5-89d9-bf95fe9ee04b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:53.234253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5007e193-cf1a-45d5-89d9-bf95fe9ee04b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:53.234703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5007e193-cf1a-45d5-89d9-bf95fe9ee04b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:53.235124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5007e193-cf1a-45d5-89d9-bf95fe9ee04b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:53.261197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5bbaea1d-c9cf-43ed-a23e-eb4f616ee599 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:53.263385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5bbaea1d-c9cf-43ed-a23e-eb4f616ee599 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:53.263870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5007e193-cf1a-45d5-89d9-bf95fe9ee04b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043737 Aug 24 10:26:53.264469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 616/1370] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:53 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:53.270651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbaea1d-c9cf-43ed-a23e-eb4f616ee599 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:53.271128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbaea1d-c9cf-43ed-a23e-eb4f616ee599 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:53.271568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbaea1d-c9cf-43ed-a23e-eb4f616ee599 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:53.272300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbaea1d-c9cf-43ed-a23e-eb4f616ee599 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:53.272760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbaea1d-c9cf-43ed-a23e-eb4f616ee599 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:53.273188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbaea1d-c9cf-43ed-a23e-eb4f616ee599 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:53.302953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5bbaea1d-c9cf-43ed-a23e-eb4f616ee599 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045759 Aug 24 10:26:53.303499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 755/1371] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:53.755420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:53.756140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:53.763366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:53.764232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:53.764674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:53.766014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:53.766618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:53.767060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:53.829491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:53.837436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-870000f2-b379-462a-81f9-efbeec76e3c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086034 Aug 24 10:26:53.838001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 617/1372] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:53 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:54.278880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bc059283-1933-4e11-bec8-c510e29ee94d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:54.281515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bc059283-1933-4e11-bec8-c510e29ee94d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:54.287963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc059283-1933-4e11-bec8-c510e29ee94d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:54.288470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc059283-1933-4e11-bec8-c510e29ee94d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:54.288904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc059283-1933-4e11-bec8-c510e29ee94d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:54.289588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc059283-1933-4e11-bec8-c510e29ee94d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:54.290034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc059283-1933-4e11-bec8-c510e29ee94d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:54.290449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc059283-1933-4e11-bec8-c510e29ee94d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:54.307356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:54.315089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-56006fae-e18b-4a17-a3e7-8fc1d7a551d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:54.317318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-56006fae-e18b-4a17-a3e7-8fc1d7a551d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:54.321017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bc059283-1933-4e11-bec8-c510e29ee94d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046056 Aug 24 10:26:54.321566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 756/1373] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:54 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:54.323597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56006fae-e18b-4a17-a3e7-8fc1d7a551d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:54.324179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56006fae-e18b-4a17-a3e7-8fc1d7a551d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:54.324631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56006fae-e18b-4a17-a3e7-8fc1d7a551d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:54.325316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56006fae-e18b-4a17-a3e7-8fc1d7a551d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:54.325762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56006fae-e18b-4a17-a3e7-8fc1d7a551d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:54.326177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56006fae-e18b-4a17-a3e7-8fc1d7a551d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:54.355648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-56006fae-e18b-4a17-a3e7-8fc1d7a551d3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044564 Aug 24 10:26:54.356224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 618/1374] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:54 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:54.858321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:54.858978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:54.864879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:54.865357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:54.865817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:54.866494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:54.866942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:54.867361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:54.927368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:54.935452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a61bb8c8-6bbf-4eb8-b383-1bda55bbba31 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081226 Aug 24 10:26:54.936011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 757/1375] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:54 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:55.334273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2120d0c5-c3ec-47e8-8013-91a84ce26b31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:55.336446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2120d0c5-c3ec-47e8-8013-91a84ce26b31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:55.343540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2120d0c5-c3ec-47e8-8013-91a84ce26b31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:55.344022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2120d0c5-c3ec-47e8-8013-91a84ce26b31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:55.344487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2120d0c5-c3ec-47e8-8013-91a84ce26b31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:55.345177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2120d0c5-c3ec-47e8-8013-91a84ce26b31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:55.345629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2120d0c5-c3ec-47e8-8013-91a84ce26b31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:55.346054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2120d0c5-c3ec-47e8-8013-91a84ce26b31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:55.368936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9f6ecb86-c06d-4b55-8dab-79bb54987e77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:55.371284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9f6ecb86-c06d-4b55-8dab-79bb54987e77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:55.375345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2120d0c5-c3ec-47e8-8013-91a84ce26b31 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045056 Aug 24 10:26:55.375920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 619/1376] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:55 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:55.378948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f6ecb86-c06d-4b55-8dab-79bb54987e77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:55.379467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f6ecb86-c06d-4b55-8dab-79bb54987e77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:55.379908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f6ecb86-c06d-4b55-8dab-79bb54987e77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:55.380646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f6ecb86-c06d-4b55-8dab-79bb54987e77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:55.381133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f6ecb86-c06d-4b55-8dab-79bb54987e77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:55.381582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f6ecb86-c06d-4b55-8dab-79bb54987e77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:55.412057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9f6ecb86-c06d-4b55-8dab-79bb54987e77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047341 Aug 24 10:26:55.412679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 758/1377] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:55 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:55.958480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:55.959099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:55.965262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:55.965770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:55.966227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:55.966909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:55.967362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:55.967813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:56.030862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:56.038548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-efbe50e1-5ab9-44ca-b6b4-d33686c3e13f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084302 Aug 24 10:26:56.039108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 620/1378] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:55 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:56.386925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f5a5d7bd-6685-4d29-b153-c471259dd297 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:56.389072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f5a5d7bd-6685-4d29-b153-c471259dd297 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:56.395973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5a5d7bd-6685-4d29-b153-c471259dd297 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:56.396478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5a5d7bd-6685-4d29-b153-c471259dd297 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:56.396910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5a5d7bd-6685-4d29-b153-c471259dd297 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:56.397592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5a5d7bd-6685-4d29-b153-c471259dd297 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:56.398039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5a5d7bd-6685-4d29-b153-c471259dd297 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:56.398463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5a5d7bd-6685-4d29-b153-c471259dd297 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:56.424344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d30c7afa-1044-438b-b85c-991da9f00ab8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:56.426570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d30c7afa-1044-438b-b85c-991da9f00ab8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:56.426802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f5a5d7bd-6685-4d29-b153-c471259dd297 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043886 Aug 24 10:26:56.427331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 759/1379] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:56 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:56.432929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d30c7afa-1044-438b-b85c-991da9f00ab8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:56.433411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d30c7afa-1044-438b-b85c-991da9f00ab8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:56.433845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d30c7afa-1044-438b-b85c-991da9f00ab8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:56.434545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d30c7afa-1044-438b-b85c-991da9f00ab8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:56.435040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d30c7afa-1044-438b-b85c-991da9f00ab8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:56.435466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d30c7afa-1044-438b-b85c-991da9f00ab8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:56.466060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d30c7afa-1044-438b-b85c-991da9f00ab8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045680 Aug 24 10:26:56.466609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 621/1380] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:56 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:57.059465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:57.060002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:57.066595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:57.067086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:57.067531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:57.068259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:57.068721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:57.069154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:57.129353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:57.137185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e07497ec-d9a8-4ad8-ada7-32adc85d42f5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081691 Aug 24 10:26:57.137736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 760/1381] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:57 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:57.438976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5f742e5c-a373-4a71-badf-591d19306b3f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:57.441170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5f742e5c-a373-4a71-badf-591d19306b3f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:57.447642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f742e5c-a373-4a71-badf-591d19306b3f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:57.448162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f742e5c-a373-4a71-badf-591d19306b3f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:57.448600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f742e5c-a373-4a71-badf-591d19306b3f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:57.449301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f742e5c-a373-4a71-badf-591d19306b3f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:57.449777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f742e5c-a373-4a71-badf-591d19306b3f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:57.450250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f742e5c-a373-4a71-badf-591d19306b3f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:57.478529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-12719096-03e5-44a9-a9e1-8cbc128830ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:57.480971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-12719096-03e5-44a9-a9e1-8cbc128830ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:57.481382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5f742e5c-a373-4a71-badf-591d19306b3f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046378 Aug 24 10:26:57.481997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 622/1382] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:57 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:57.487934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12719096-03e5-44a9-a9e1-8cbc128830ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:57.488444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12719096-03e5-44a9-a9e1-8cbc128830ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:57.488907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12719096-03e5-44a9-a9e1-8cbc128830ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:57.489600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12719096-03e5-44a9-a9e1-8cbc128830ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:57.490054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12719096-03e5-44a9-a9e1-8cbc128830ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:57.490477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12719096-03e5-44a9-a9e1-8cbc128830ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:57.522079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-12719096-03e5-44a9-a9e1-8cbc128830ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047529 Aug 24 10:26:57.522636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 761/1383] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:57 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:58.158393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:58.158950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:58.165116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:58.165603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:58.166041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:58.166787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:58.167252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:58.167689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:58.227612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:58.235877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bfc5863e-881e-4173-9a91-8759a6ab3131 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082135 Aug 24 10:26:58.236474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 623/1384] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:58 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:26:58.494808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0027e789-80a7-4200-b0df-8d7b342916bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:58.496985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0027e789-80a7-4200-b0df-8d7b342916bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:58.503502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0027e789-80a7-4200-b0df-8d7b342916bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:58.503990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0027e789-80a7-4200-b0df-8d7b342916bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:58.504463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0027e789-80a7-4200-b0df-8d7b342916bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:58.505157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0027e789-80a7-4200-b0df-8d7b342916bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:58.505616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0027e789-80a7-4200-b0df-8d7b342916bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:58.506047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0027e789-80a7-4200-b0df-8d7b342916bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:58.535030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dc2ad981-4690-400b-8e45-e1d89c8cf004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:58.536273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0027e789-80a7-4200-b0df-8d7b342916bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045586 Aug 24 10:26:58.536824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 762/1385] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:58 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:26:58.537177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dc2ad981-4690-400b-8e45-e1d89c8cf004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:58.543371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc2ad981-4690-400b-8e45-e1d89c8cf004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:58.543840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc2ad981-4690-400b-8e45-e1d89c8cf004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:58.544309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc2ad981-4690-400b-8e45-e1d89c8cf004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:58.544995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc2ad981-4690-400b-8e45-e1d89c8cf004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:58.545444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc2ad981-4690-400b-8e45-e1d89c8cf004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:58.545890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc2ad981-4690-400b-8e45-e1d89c8cf004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:58.574848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dc2ad981-4690-400b-8e45-e1d89c8cf004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044364 Aug 24 10:26:58.575396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 624/1386] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:58 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:59.255905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:59.256547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:59.262540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:59.263027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:59.263467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:59.264204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:59.264662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:59.265090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:59.327652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:26:59.335380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-89ac22ad-af93-4f07-ac41-f964e3661ad7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083332 Aug 24 10:26:59.335933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 763/1387] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:26:59 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:26:59.549703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4b84e436-b31d-4520-b3c5-d4d0ecb881f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:59.551758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4b84e436-b31d-4520-b3c5-d4d0ecb881f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:59.558279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b84e436-b31d-4520-b3c5-d4d0ecb881f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:59.558766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b84e436-b31d-4520-b3c5-d4d0ecb881f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:59.559205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b84e436-b31d-4520-b3c5-d4d0ecb881f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:59.559898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b84e436-b31d-4520-b3c5-d4d0ecb881f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:59.560383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b84e436-b31d-4520-b3c5-d4d0ecb881f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:59.560811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b84e436-b31d-4520-b3c5-d4d0ecb881f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:59.587855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1e4dd6bd-aab5-47df-8e2a-1e9f76d453c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:26:59.590014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1e4dd6bd-aab5-47df-8e2a-1e9f76d453c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:26:59.590626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4b84e436-b31d-4520-b3c5-d4d0ecb881f6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045497 Aug 24 10:26:59.591186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 625/1388] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:59 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:26:59.596963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e4dd6bd-aab5-47df-8e2a-1e9f76d453c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:59.597435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e4dd6bd-aab5-47df-8e2a-1e9f76d453c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:59.597907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e4dd6bd-aab5-47df-8e2a-1e9f76d453c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:59.598591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e4dd6bd-aab5-47df-8e2a-1e9f76d453c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:26:59.599041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e4dd6bd-aab5-47df-8e2a-1e9f76d453c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:26:59.599462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e4dd6bd-aab5-47df-8e2a-1e9f76d453c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:26:59.628220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1e4dd6bd-aab5-47df-8e2a-1e9f76d453c7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044236 Aug 24 10:26:59.628754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 764/1389] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:26:59 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:00.355767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:00.356408 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:00.362341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:00.362820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:00.363255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:00.363937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:00.364415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:00.364834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:00.424564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:00.432160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3e5626a9-cf8d-471b-a2c2-51b720bd9c6b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080304 Aug 24 10:27:00.432830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 626/1390] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:00 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:00.603467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-03adee86-c82c-4a71-9b7d-71e65eb485ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:00.605670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-03adee86-c82c-4a71-9b7d-71e65eb485ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:00.612673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03adee86-c82c-4a71-9b7d-71e65eb485ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:00.613072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03adee86-c82c-4a71-9b7d-71e65eb485ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:00.613521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03adee86-c82c-4a71-9b7d-71e65eb485ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:00.614211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03adee86-c82c-4a71-9b7d-71e65eb485ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:00.614673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03adee86-c82c-4a71-9b7d-71e65eb485ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:00.615107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03adee86-c82c-4a71-9b7d-71e65eb485ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:00.640424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-52607ece-2744-4e46-8314-19be570ffea0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:00.642558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-52607ece-2744-4e46-8314-19be570ffea0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:00.644341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-03adee86-c82c-4a71-9b7d-71e65eb485ca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044876 Aug 24 10:27:00.644899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 765/1391] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:00 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:00.649068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52607ece-2744-4e46-8314-19be570ffea0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:00.649555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52607ece-2744-4e46-8314-19be570ffea0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:00.649994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52607ece-2744-4e46-8314-19be570ffea0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:00.650694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52607ece-2744-4e46-8314-19be570ffea0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:00.651153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52607ece-2744-4e46-8314-19be570ffea0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:00.651587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52607ece-2744-4e46-8314-19be570ffea0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:00.681129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-52607ece-2744-4e46-8314-19be570ffea0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044607 Aug 24 10:27:00.681662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 627/1392] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:00 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:01.454038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:01.454660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:01.460958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:01.461452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:01.462081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:01.462775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:01.463236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:01.463896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:01.527338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:01.535850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-392140c6-3a94-40d9-9031-14d7f0a3abcb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086462 Aug 24 10:27:01.540195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 766/1393] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:01 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:01.657163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-89b75673-db78-4d31-9d49-a63d8c5ba56e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:01.659488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-89b75673-db78-4d31-9d49-a63d8c5ba56e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:01.666471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b75673-db78-4d31-9d49-a63d8c5ba56e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:01.667043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b75673-db78-4d31-9d49-a63d8c5ba56e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:01.667521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b75673-db78-4d31-9d49-a63d8c5ba56e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:01.668289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b75673-db78-4d31-9d49-a63d8c5ba56e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:01.668828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b75673-db78-4d31-9d49-a63d8c5ba56e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:01.669328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b75673-db78-4d31-9d49-a63d8c5ba56e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:01.673275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:01.693287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-95dd559c-ca7b-402f-bb5d-0fee18447b3c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:01.695412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-95dd559c-ca7b-402f-bb5d-0fee18447b3c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:01.702486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-89b75673-db78-4d31-9d49-a63d8c5ba56e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048496 Aug 24 10:27:01.702653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95dd559c-ca7b-402f-bb5d-0fee18447b3c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:01.703366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95dd559c-ca7b-402f-bb5d-0fee18447b3c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:01.703506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 628/1394] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:01 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:01.703631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95dd559c-ca7b-402f-bb5d-0fee18447b3c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:01.704267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95dd559c-ca7b-402f-bb5d-0fee18447b3c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:01.704719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95dd559c-ca7b-402f-bb5d-0fee18447b3c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:01.705142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95dd559c-ca7b-402f-bb5d-0fee18447b3c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:01.734001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-95dd559c-ca7b-402f-bb5d-0fee18447b3c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044684 Aug 24 10:27:01.734543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 767/1395] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:01 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:02.556581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:02.557217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:02.563098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:02.563640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:02.564101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:02.564784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:02.565233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:02.565656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:02.625615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:02.633210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4d3fecd9-91e3-4fb9-8c52-ac264cfb0563 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080528 Aug 24 10:27:02.633957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 629/1396] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:02 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:02.714906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ecb24a24-d36c-4dcc-a52c-9e55edd33c5b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:02.717039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ecb24a24-d36c-4dcc-a52c-9e55edd33c5b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:02.723894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecb24a24-d36c-4dcc-a52c-9e55edd33c5b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:02.724398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecb24a24-d36c-4dcc-a52c-9e55edd33c5b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:02.724920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecb24a24-d36c-4dcc-a52c-9e55edd33c5b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:02.725607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecb24a24-d36c-4dcc-a52c-9e55edd33c5b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:02.726057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecb24a24-d36c-4dcc-a52c-9e55edd33c5b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:02.726477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecb24a24-d36c-4dcc-a52c-9e55edd33c5b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:02.746520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4b36feff-c514-4661-9d3d-b248be69d888 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:02.748682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4b36feff-c514-4661-9d3d-b248be69d888 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:02.754974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b36feff-c514-4661-9d3d-b248be69d888 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:02.755389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ecb24a24-d36c-4dcc-a52c-9e55edd33c5b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044379 Aug 24 10:27:02.755544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b36feff-c514-4661-9d3d-b248be69d888 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:02.755871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b36feff-c514-4661-9d3d-b248be69d888 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:02.756333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 768/1397] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:02 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:02.756664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b36feff-c514-4661-9d3d-b248be69d888 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:02.757181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b36feff-c514-4661-9d3d-b248be69d888 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:02.757607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b36feff-c514-4661-9d3d-b248be69d888 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:02.788090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4b36feff-c514-4661-9d3d-b248be69d888 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045415 Aug 24 10:27:02.788659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 630/1398] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:02 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:03.653950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:03.654480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:03.660458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:03.660949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:03.661391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:03.662080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:03.662546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:03.662969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:03.723420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:03.731567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-352586ed-98e2-4954-9325-4893ec5cd257 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081596 Aug 24 10:27:03.732141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 769/1399] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:03 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:03.767548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d56ea784-3d1d-4cfd-a7c4-f41b257aca94 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:03.769699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d56ea784-3d1d-4cfd-a7c4-f41b257aca94 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:03.776205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d56ea784-3d1d-4cfd-a7c4-f41b257aca94 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:03.776681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d56ea784-3d1d-4cfd-a7c4-f41b257aca94 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:03.777106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d56ea784-3d1d-4cfd-a7c4-f41b257aca94 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:03.777788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d56ea784-3d1d-4cfd-a7c4-f41b257aca94 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:03.778238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d56ea784-3d1d-4cfd-a7c4-f41b257aca94 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:03.778655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d56ea784-3d1d-4cfd-a7c4-f41b257aca94 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:03.808069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d56ea784-3d1d-4cfd-a7c4-f41b257aca94 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044359 Aug 24 10:27:03.808633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 631/1400] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:03 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:03.861925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-343b9888-04d8-43ba-a4db-9681e72a0f1f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:03.864071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-343b9888-04d8-43ba-a4db-9681e72a0f1f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:03.869959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-343b9888-04d8-43ba-a4db-9681e72a0f1f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:03.870447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-343b9888-04d8-43ba-a4db-9681e72a0f1f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:03.870890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-343b9888-04d8-43ba-a4db-9681e72a0f1f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:03.871581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-343b9888-04d8-43ba-a4db-9681e72a0f1f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:03.872065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-343b9888-04d8-43ba-a4db-9681e72a0f1f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:03.872498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-343b9888-04d8-43ba-a4db-9681e72a0f1f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:03.901429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-343b9888-04d8-43ba-a4db-9681e72a0f1f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.105138 Aug 24 10:27:03.901973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 770/1401] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:03 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:04.752989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:04.753593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:04.759404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:04.759878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:04.760338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:04.761015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:04.761456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:04.761928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:04.828286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:04.836105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-af184dda-4183-4e50-9813-4cf3ef4d3e51 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087423 Aug 24 10:27:04.836687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 632/1402] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:04.885474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-908c0726-2bee-4af2-9a8a-74d98dcdae46 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:27:04.887735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-908c0726-2bee-4af2-9a8a-74d98dcdae46 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:04.894384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-908c0726-2bee-4af2-9a8a-74d98dcdae46 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:04.894873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-908c0726-2bee-4af2-9a8a-74d98dcdae46 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:04.895314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-908c0726-2bee-4af2-9a8a-74d98dcdae46 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:04.896011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-908c0726-2bee-4af2-9a8a-74d98dcdae46 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:04.896499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-908c0726-2bee-4af2-9a8a-74d98dcdae46 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:04.896930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-908c0726-2bee-4af2-9a8a-74d98dcdae46 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:04.914098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fffb52fd-c195-415f-9130-9d6e7f3672ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:04.916247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fffb52fd-c195-415f-9130-9d6e7f3672ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:04.922204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fffb52fd-c195-415f-9130-9d6e7f3672ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:04.922884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fffb52fd-c195-415f-9130-9d6e7f3672ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:04.923333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fffb52fd-c195-415f-9130-9d6e7f3672ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:04.924210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fffb52fd-c195-415f-9130-9d6e7f3672ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:04.924667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fffb52fd-c195-415f-9130-9d6e7f3672ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:04.925091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fffb52fd-c195-415f-9130-9d6e7f3672ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:04.955010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fffb52fd-c195-415f-9130-9d6e7f3672ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044843 Aug 24 10:27:04.955564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 633/1403] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:04 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:04.993191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-908c0726-2bee-4af2-9a8a-74d98dcdae46 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "POST /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/action" status: 200 len: 76050 microversion: 2.1 time: 0.176025 Aug 24 10:27:04.993750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 771/1404] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:27:04 2022] POST /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1/action => generated 76050 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 24 10:27:05.856500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:05.857132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:05.863384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:05.863854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:05.864320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:05.864995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:05.865440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:05.865855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:05.925015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:05.932524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f7bc1c86-23fb-4383-b253-a090b9e8a19b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.079816 Aug 24 10:27:05.933078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 634/1405] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:05.967912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b475bbd0-a3e6-43a2-82ab-78ac724492bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:05.970207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b475bbd0-a3e6-43a2-82ab-78ac724492bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:05.977228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b475bbd0-a3e6-43a2-82ab-78ac724492bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:05.977731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b475bbd0-a3e6-43a2-82ab-78ac724492bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:05.978161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b475bbd0-a3e6-43a2-82ab-78ac724492bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:05.978856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b475bbd0-a3e6-43a2-82ab-78ac724492bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:05.979317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b475bbd0-a3e6-43a2-82ab-78ac724492bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:05.979750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b475bbd0-a3e6-43a2-82ab-78ac724492bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:06.009649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b475bbd0-a3e6-43a2-82ab-78ac724492bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046612 Aug 24 10:27:06.010203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 772/1406] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:05 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:06.954117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:06.954728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:06.961208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:06.961688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:06.962121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:06.962804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:06.963269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:06.963682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:07.022816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-67564756-2848-49d8-beaa-c14dfee64ae2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:07.024912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:07.025073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-67564756-2848-49d8-beaa-c14dfee64ae2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:07.031472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67564756-2848-49d8-beaa-c14dfee64ae2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:07.031961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67564756-2848-49d8-beaa-c14dfee64ae2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:07.032431 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67564756-2848-49d8-beaa-c14dfee64ae2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:07.032704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fc87c99c-ebf5-4125-8199-e385efb61c54 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083338 Aug 24 10:27:07.033135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67564756-2848-49d8-beaa-c14dfee64ae2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:07.033247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 635/1407] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:06 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:07.033595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67564756-2848-49d8-beaa-c14dfee64ae2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:07.034033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-67564756-2848-49d8-beaa-c14dfee64ae2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:07.051964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:07.065921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-67564756-2848-49d8-beaa-c14dfee64ae2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047269 Aug 24 10:27:07.066468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 773/1408] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:07 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:08.053451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:08.054019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:08.060371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:08.060848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:08.061278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:08.061961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:08.062409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:08.062829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:08.079059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f1d89518-a7a3-44ea-912f-7d711e624a2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:08.081212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f1d89518-a7a3-44ea-912f-7d711e624a2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:08.087592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1d89518-a7a3-44ea-912f-7d711e624a2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:08.088203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1d89518-a7a3-44ea-912f-7d711e624a2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:08.088658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1d89518-a7a3-44ea-912f-7d711e624a2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:08.089493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1d89518-a7a3-44ea-912f-7d711e624a2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:08.089988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1d89518-a7a3-44ea-912f-7d711e624a2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:08.090420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1d89518-a7a3-44ea-912f-7d711e624a2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:08.120658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f1d89518-a7a3-44ea-912f-7d711e624a2a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046235 Aug 24 10:27:08.121436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 774/1409] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:08 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:08.123952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:08.131636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c6f7e8d5-1d49-4b64-be2b-56bd29436453 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082199 Aug 24 10:27:08.132207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 636/1410] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:08 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:09.132255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9cb0a828-1155-490e-8e08-1c5e67fbc2c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:09.134390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9cb0a828-1155-490e-8e08-1c5e67fbc2c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:09.140810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cb0a828-1155-490e-8e08-1c5e67fbc2c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:09.141296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cb0a828-1155-490e-8e08-1c5e67fbc2c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:09.141740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cb0a828-1155-490e-8e08-1c5e67fbc2c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:09.142432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cb0a828-1155-490e-8e08-1c5e67fbc2c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:09.142892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cb0a828-1155-490e-8e08-1c5e67fbc2c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:09.143320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cb0a828-1155-490e-8e08-1c5e67fbc2c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:09.152220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:09.152844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:09.160002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:09.160522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:09.160957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:09.161647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:09.162101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:09.162526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:09.173626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9cb0a828-1155-490e-8e08-1c5e67fbc2c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045310 Aug 24 10:27:09.174186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 775/1411] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:09 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:09.226798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:09.235006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-de9b3fa2-e274-4bc0-a999-80d631324476 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086785 Aug 24 10:27:09.235575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 637/1412] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:09 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:27:10.186023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a94d9175-b29b-4d0d-acab-0c578e0a8ff8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:10.188906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a94d9175-b29b-4d0d-acab-0c578e0a8ff8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:10.195106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a94d9175-b29b-4d0d-acab-0c578e0a8ff8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:10.195596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a94d9175-b29b-4d0d-acab-0c578e0a8ff8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:10.196038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a94d9175-b29b-4d0d-acab-0c578e0a8ff8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:10.196763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a94d9175-b29b-4d0d-acab-0c578e0a8ff8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:10.197226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a94d9175-b29b-4d0d-acab-0c578e0a8ff8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:10.197688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a94d9175-b29b-4d0d-acab-0c578e0a8ff8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:10.227815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a94d9175-b29b-4d0d-acab-0c578e0a8ff8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045791 Aug 24 10:27:10.228385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 776/1413] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:10 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:10.263805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:10.264949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:10.271289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:10.271758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:10.272218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:10.272895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:10.273341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:10.273752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:10.333669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:10.341451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a9711021-89d2-45e9-8111-3d3dea29581c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081513 Aug 24 10:27:10.342007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 638/1414] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:10 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:11.240483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3db8f2d1-c6b6-4234-88bc-93e7dca03a97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:11.242640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3db8f2d1-c6b6-4234-88bc-93e7dca03a97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:11.249792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3db8f2d1-c6b6-4234-88bc-93e7dca03a97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:11.250276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3db8f2d1-c6b6-4234-88bc-93e7dca03a97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:11.250718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3db8f2d1-c6b6-4234-88bc-93e7dca03a97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:11.251412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3db8f2d1-c6b6-4234-88bc-93e7dca03a97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:11.251864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3db8f2d1-c6b6-4234-88bc-93e7dca03a97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:11.252381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3db8f2d1-c6b6-4234-88bc-93e7dca03a97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:11.282458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3db8f2d1-c6b6-4234-88bc-93e7dca03a97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045972 Aug 24 10:27:11.283020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 777/1415] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:11 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:11.363469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:11.364094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:11.370548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:11.371042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:11.371479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:11.372216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:11.372673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:11.373097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:11.437268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:11.445489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-06658d49-ee9f-4a21-a0df-f49fb70e1285 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085922 Aug 24 10:27:11.446050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 639/1416] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:11 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:12.295169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-aae90229-541a-4a11-a7fb-fbdc55ca513f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:12.297386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-aae90229-541a-4a11-a7fb-fbdc55ca513f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:12.303811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aae90229-541a-4a11-a7fb-fbdc55ca513f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:12.304322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aae90229-541a-4a11-a7fb-fbdc55ca513f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:12.304763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aae90229-541a-4a11-a7fb-fbdc55ca513f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:12.305448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aae90229-541a-4a11-a7fb-fbdc55ca513f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:12.305901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aae90229-541a-4a11-a7fb-fbdc55ca513f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:12.306323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aae90229-541a-4a11-a7fb-fbdc55ca513f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:12.335655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-aae90229-541a-4a11-a7fb-fbdc55ca513f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044971 Aug 24 10:27:12.336222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 778/1417] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:12 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:12.466359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:12.466976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:12.473944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:12.474418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:12.474855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:12.475531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:12.475980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:12.476422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:12.536981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:12.544711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5369b075-807a-45a9-9646-66bded17d18c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082299 Aug 24 10:27:12.545337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 640/1418] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:12 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:13.348759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7a6feb00-df9c-49b9-a81d-ef14e3f95930 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:13.350813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7a6feb00-df9c-49b9-a81d-ef14e3f95930 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:13.357243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a6feb00-df9c-49b9-a81d-ef14e3f95930 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:13.357730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a6feb00-df9c-49b9-a81d-ef14e3f95930 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:13.358201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a6feb00-df9c-49b9-a81d-ef14e3f95930 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:13.358892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a6feb00-df9c-49b9-a81d-ef14e3f95930 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:13.359353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a6feb00-df9c-49b9-a81d-ef14e3f95930 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:13.359783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a6feb00-df9c-49b9-a81d-ef14e3f95930 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:13.387874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7a6feb00-df9c-49b9-a81d-ef14e3f95930 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043167 Aug 24 10:27:13.388449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 779/1419] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:13 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:13.566056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:13.566666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:13.573061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:13.573531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:13.573960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:13.574634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:13.575077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:13.575494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:13.642175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:13.649851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-809c6bf3-1aec-4ebd-a959-4d99435f5f1d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087654 Aug 24 10:27:13.650411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 641/1420] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:13 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:14.400629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b9e95203-16c7-4700-b578-7c4634a32321 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:14.402801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b9e95203-16c7-4700-b578-7c4634a32321 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:14.409762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9e95203-16c7-4700-b578-7c4634a32321 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:14.410243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9e95203-16c7-4700-b578-7c4634a32321 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:14.410678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9e95203-16c7-4700-b578-7c4634a32321 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:14.411369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9e95203-16c7-4700-b578-7c4634a32321 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:14.411822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9e95203-16c7-4700-b578-7c4634a32321 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:14.412279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9e95203-16c7-4700-b578-7c4634a32321 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:14.441173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b9e95203-16c7-4700-b578-7c4634a32321 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044545 Aug 24 10:27:14.441721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 780/1421] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:14 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:14.670305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:14.671517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:14.678192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:14.678671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:14.679103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:14.679820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:14.680306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:14.680733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:14.741240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:14.749060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-41b29616-d276-4cff-8d81-c36ad0e10f94 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082750 Aug 24 10:27:14.749623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 642/1422] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:14 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:15.454261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5ba3cd25-f1f5-4c26-99dc-dabc5697b969 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:15.456427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5ba3cd25-f1f5-4c26-99dc-dabc5697b969 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:15.462725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ba3cd25-f1f5-4c26-99dc-dabc5697b969 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:15.463204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ba3cd25-f1f5-4c26-99dc-dabc5697b969 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:15.463640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ba3cd25-f1f5-4c26-99dc-dabc5697b969 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:15.464361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ba3cd25-f1f5-4c26-99dc-dabc5697b969 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:15.464816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ba3cd25-f1f5-4c26-99dc-dabc5697b969 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:15.465249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ba3cd25-f1f5-4c26-99dc-dabc5697b969 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:15.494342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5ba3cd25-f1f5-4c26-99dc-dabc5697b969 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044867 Aug 24 10:27:15.494815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 781/1423] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:15 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:15.769528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:15.770064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:15.776622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:15.777100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:15.777539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:15.778237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:15.778693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:15.779116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:15.840158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:15.848175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-653f2d5c-b37b-4da9-aee9-3eb87aaa20eb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082608 Aug 24 10:27:15.848725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 643/1424] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:15 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:16.507058 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b5cb888b-abe0-4bfc-82a8-8ca61b0e3348 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:16.509208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b5cb888b-abe0-4bfc-82a8-8ca61b0e3348 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:16.516276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5cb888b-abe0-4bfc-82a8-8ca61b0e3348 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:16.516760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5cb888b-abe0-4bfc-82a8-8ca61b0e3348 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:16.517198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5cb888b-abe0-4bfc-82a8-8ca61b0e3348 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:16.517887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5cb888b-abe0-4bfc-82a8-8ca61b0e3348 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:16.518345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5cb888b-abe0-4bfc-82a8-8ca61b0e3348 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:16.518774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5cb888b-abe0-4bfc-82a8-8ca61b0e3348 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:16.547709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b5cb888b-abe0-4bfc-82a8-8ca61b0e3348 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044590 Aug 24 10:27:16.548278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 782/1425] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:16 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:16.869141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:16.869753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:16.876664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:16.877137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:16.877574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:16.878255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:16.878704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:16.879123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:16.938769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:16.942247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:16.947403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8de27a5a-500a-434e-bbb4-9c983dc19940 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082091 Aug 24 10:27:16.947962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 644/1426] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:16 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:17.560123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d2dd1464-b2b1-4989-ba8e-47110a29b403 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:17.562675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d2dd1464-b2b1-4989-ba8e-47110a29b403 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:17.569023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2dd1464-b2b1-4989-ba8e-47110a29b403 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:17.569509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2dd1464-b2b1-4989-ba8e-47110a29b403 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:17.569953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2dd1464-b2b1-4989-ba8e-47110a29b403 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:17.570644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2dd1464-b2b1-4989-ba8e-47110a29b403 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:17.571105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2dd1464-b2b1-4989-ba8e-47110a29b403 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:17.571537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d2dd1464-b2b1-4989-ba8e-47110a29b403 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:17.601134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d2dd1464-b2b1-4989-ba8e-47110a29b403 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044987 Aug 24 10:27:17.601681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 783/1427] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:17 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:17.968184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:17.968793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:17.975103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:17.975580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:17.976012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:17.976723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:17.977167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:17.977586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:18.040551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:18.049222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a6dd571c-830c-4283-be5a-f411d74af073 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085182 Aug 24 10:27:18.049786 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 645/1428] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:17 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:18.613363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6a9e8b26-dc8c-40c2-a244-d169b185ae86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:18.615508 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6a9e8b26-dc8c-40c2-a244-d169b185ae86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:18.622180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6a9e8b26-dc8c-40c2-a244-d169b185ae86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:18.622662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6a9e8b26-dc8c-40c2-a244-d169b185ae86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:18.623101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6a9e8b26-dc8c-40c2-a244-d169b185ae86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:18.623781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6a9e8b26-dc8c-40c2-a244-d169b185ae86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:18.624409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6a9e8b26-dc8c-40c2-a244-d169b185ae86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:18.624838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6a9e8b26-dc8c-40c2-a244-d169b185ae86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:18.653898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6a9e8b26-dc8c-40c2-a244-d169b185ae86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044502 Aug 24 10:27:18.654443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 784/1429] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:18 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:19.069433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:19.070044 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:19.076691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:19.077171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:19.078250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:19.078936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:19.079391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:19.079816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:19.139451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:19.147713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-af579237-e966-463c-a770-468092ec6fd5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082218 Aug 24 10:27:19.148305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 646/1430] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:19 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:19.666853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8af1742b-5223-4847-abab-8509e557e056 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:19.669012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8af1742b-5223-4847-abab-8509e557e056 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:19.675878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8af1742b-5223-4847-abab-8509e557e056 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:19.676387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8af1742b-5223-4847-abab-8509e557e056 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:19.676823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8af1742b-5223-4847-abab-8509e557e056 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:19.677511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8af1742b-5223-4847-abab-8509e557e056 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:19.677961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8af1742b-5223-4847-abab-8509e557e056 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:19.678380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8af1742b-5223-4847-abab-8509e557e056 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:19.707263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8af1742b-5223-4847-abab-8509e557e056 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044330 Aug 24 10:27:19.707811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 785/1431] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:19 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:20.168441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:20.169073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:20.171004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=70 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:20.176343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:20.176829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:20.177266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:20.177961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:20.178412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:20.178837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:20.240430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:20.248095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9d10cac7-e982-4db1-b09c-d5b31bc800fb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083777 Aug 24 10:27:20.248673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 647/1432] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:20 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:20.718827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a2f1ac52-97cd-4542-ba27-aeaff6c03eb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:20.721028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a2f1ac52-97cd-4542-ba27-aeaff6c03eb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:20.727388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f1ac52-97cd-4542-ba27-aeaff6c03eb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:20.727867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f1ac52-97cd-4542-ba27-aeaff6c03eb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:20.728332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f1ac52-97cd-4542-ba27-aeaff6c03eb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:20.729012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f1ac52-97cd-4542-ba27-aeaff6c03eb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:20.729462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f1ac52-97cd-4542-ba27-aeaff6c03eb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:20.729883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2f1ac52-97cd-4542-ba27-aeaff6c03eb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:20.759286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a2f1ac52-97cd-4542-ba27-aeaff6c03eb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044374 Aug 24 10:27:20.759822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 786/1433] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:20 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:21.268919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:21.269523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:21.276632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:21.277135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:21.277563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:21.278241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:21.278687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:21.279105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:21.341522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:21.349568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fa80c62e-b359-4cb4-8d50-69a3082f67b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084610 Aug 24 10:27:21.350149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 648/1434] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:21 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:21.771724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1ca60d11-ef24-4f67-8d57-29d4f56c462c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:21.773917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1ca60d11-ef24-4f67-8d57-29d4f56c462c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:21.776250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=70 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:21.782024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ca60d11-ef24-4f67-8d57-29d4f56c462c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:21.782539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ca60d11-ef24-4f67-8d57-29d4f56c462c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:21.782982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ca60d11-ef24-4f67-8d57-29d4f56c462c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:21.783679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ca60d11-ef24-4f67-8d57-29d4f56c462c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:21.784162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ca60d11-ef24-4f67-8d57-29d4f56c462c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:21.784591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ca60d11-ef24-4f67-8d57-29d4f56c462c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:21.813702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1ca60d11-ef24-4f67-8d57-29d4f56c462c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045960 Aug 24 10:27:21.814250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 787/1435] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:21 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:22.370736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:22.371331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:22.378414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:22.378893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:22.379323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:22.380003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:22.380481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:22.380903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:22.443800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:22.451725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-607ca64b-f58d-413c-9571-062ef97e93da tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084984 Aug 24 10:27:22.452357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 649/1436] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:22 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:22.826420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-65d1d458-daf4-4605-b554-a9d395a8bbe2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:22.828565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-65d1d458-daf4-4605-b554-a9d395a8bbe2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:22.834898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65d1d458-daf4-4605-b554-a9d395a8bbe2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:22.835379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65d1d458-daf4-4605-b554-a9d395a8bbe2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:22.835815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65d1d458-daf4-4605-b554-a9d395a8bbe2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:22.836538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65d1d458-daf4-4605-b554-a9d395a8bbe2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:22.837004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65d1d458-daf4-4605-b554-a9d395a8bbe2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:22.837442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-65d1d458-daf4-4605-b554-a9d395a8bbe2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:22.866852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-65d1d458-daf4-4605-b554-a9d395a8bbe2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044701 Aug 24 10:27:22.867401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 788/1437] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:22 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:23.472177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:23.472799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:23.479350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:23.479828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:23.480291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:23.480975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:23.481429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:23.481852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:23.541611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:23.549228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-70c79d7c-72c6-456c-8d18-ebfcc71e5b75 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080937 Aug 24 10:27:23.549783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 650/1438] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:23 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:23.879257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4968bd1d-c321-462d-a813-c2ed2b0e5520 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:23.881414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4968bd1d-c321-462d-a813-c2ed2b0e5520 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:23.887824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4968bd1d-c321-462d-a813-c2ed2b0e5520 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:23.888342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4968bd1d-c321-462d-a813-c2ed2b0e5520 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:23.888783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4968bd1d-c321-462d-a813-c2ed2b0e5520 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:23.889503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4968bd1d-c321-462d-a813-c2ed2b0e5520 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:23.889957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4968bd1d-c321-462d-a813-c2ed2b0e5520 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:23.890386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4968bd1d-c321-462d-a813-c2ed2b0e5520 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:23.918517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4968bd1d-c321-462d-a813-c2ed2b0e5520 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043269 Aug 24 10:27:23.919059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 789/1439] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:23 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:24.569420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:24.570056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:24.577234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:24.577718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:24.578144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:24.578818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:24.579264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:24.579679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:24.644514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:24.652729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-eb64a201-f6fc-4ab2-8600-807db1f5ff72 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087136 Aug 24 10:27:24.653321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 651/1440] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:24 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:24.931579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fb93cc2e-37ce-41a8-afe1-348c53395e63 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:24.933760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fb93cc2e-37ce-41a8-afe1-348c53395e63 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:24.940659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb93cc2e-37ce-41a8-afe1-348c53395e63 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:24.941140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb93cc2e-37ce-41a8-afe1-348c53395e63 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:24.941579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb93cc2e-37ce-41a8-afe1-348c53395e63 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:24.942265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb93cc2e-37ce-41a8-afe1-348c53395e63 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:24.942748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb93cc2e-37ce-41a8-afe1-348c53395e63 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:24.943177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb93cc2e-37ce-41a8-afe1-348c53395e63 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:24.946829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:24.979889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fb93cc2e-37ce-41a8-afe1-348c53395e63 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051991 Aug 24 10:27:24.980463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 790/1441] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:24 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:25.674852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:25.675425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:25.681799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:25.682272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:25.682699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:25.683397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:25.683814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:25.684262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:25.744240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:25.751958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b222f3a9-3943-45ee-a25b-a2c25787889f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081589 Aug 24 10:27:25.752560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 652/1442] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:25 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:25.992379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-42008eee-833f-4c9d-914b-246469a5f049 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:25.994542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-42008eee-833f-4c9d-914b-246469a5f049 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:26.000878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42008eee-833f-4c9d-914b-246469a5f049 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:26.001365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42008eee-833f-4c9d-914b-246469a5f049 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:26.001806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42008eee-833f-4c9d-914b-246469a5f049 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:26.002496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42008eee-833f-4c9d-914b-246469a5f049 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:26.002957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42008eee-833f-4c9d-914b-246469a5f049 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:26.003386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42008eee-833f-4c9d-914b-246469a5f049 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:26.033008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-42008eee-833f-4c9d-914b-246469a5f049 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044896 Aug 24 10:27:26.033565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 791/1443] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:25 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:26.772200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:26.772818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:26.779257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:26.779739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:26.780209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:26.780904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:26.781357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:26.781782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:26.845801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:26.854039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5d65590f-678e-469d-a22c-c78481b1c570 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085702 Aug 24 10:27:26.854608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 653/1444] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:26 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:27.044926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e116d9ce-fd75-4fad-bd98-2d72cdd249a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:27.047513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e116d9ce-fd75-4fad-bd98-2d72cdd249a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:27.054422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e116d9ce-fd75-4fad-bd98-2d72cdd249a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:27.054915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e116d9ce-fd75-4fad-bd98-2d72cdd249a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:27.055360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e116d9ce-fd75-4fad-bd98-2d72cdd249a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:27.056080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e116d9ce-fd75-4fad-bd98-2d72cdd249a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:27.056544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e116d9ce-fd75-4fad-bd98-2d72cdd249a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:27.056978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e116d9ce-fd75-4fad-bd98-2d72cdd249a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:27.086792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e116d9ce-fd75-4fad-bd98-2d72cdd249a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045946 Aug 24 10:27:27.087518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 792/1445] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:27 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:27.875324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:27.875941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:27.882340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:27.882822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:27.883260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:27.883949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:27.884443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:27.884871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:27.944901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:27.953473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ccd3a0bf-ab6a-40cf-8c2b-5b19e37c2c93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082637 Aug 24 10:27:27.954041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 654/1446] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:27 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:28.099045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-86299d64-563b-4ecc-a972-683b8466ab86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:28.101170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-86299d64-563b-4ecc-a972-683b8466ab86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:28.107608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86299d64-563b-4ecc-a972-683b8466ab86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:28.108135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86299d64-563b-4ecc-a972-683b8466ab86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:28.108589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86299d64-563b-4ecc-a972-683b8466ab86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:28.109276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86299d64-563b-4ecc-a972-683b8466ab86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:28.109738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86299d64-563b-4ecc-a972-683b8466ab86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:28.110177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86299d64-563b-4ecc-a972-683b8466ab86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:28.139789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-86299d64-563b-4ecc-a972-683b8466ab86 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044729 Aug 24 10:27:28.140965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 793/1447] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:28 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:28.974587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:28.975209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:28.981622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:28.982107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:28.982546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:28.983237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:28.983691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:28.984144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:29.046424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:29.054012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4831a455-2f9c-49ff-85ce-7caaf775025d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083360 Aug 24 10:27:29.054568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 655/1448] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:28 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:29.152521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4b6012fe-0b7b-4b12-8f2a-ce795987880e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:29.154645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4b6012fe-0b7b-4b12-8f2a-ce795987880e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:29.161593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b6012fe-0b7b-4b12-8f2a-ce795987880e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:29.162075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b6012fe-0b7b-4b12-8f2a-ce795987880e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:29.162516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b6012fe-0b7b-4b12-8f2a-ce795987880e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:29.163206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b6012fe-0b7b-4b12-8f2a-ce795987880e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:29.163659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b6012fe-0b7b-4b12-8f2a-ce795987880e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:29.164107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b6012fe-0b7b-4b12-8f2a-ce795987880e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:29.192186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4b6012fe-0b7b-4b12-8f2a-ce795987880e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044199 Aug 24 10:27:29.192738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 794/1449] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:29 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:30.074515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:30.075161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:30.081832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:30.082314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:30.082755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:30.083446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:30.083902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:30.084360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:30.147503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:30.156627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9c23a393-dd6b-444e-8f0b-3a90df49831c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086035 Aug 24 10:27:30.157197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 656/1450] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:30 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:30.204806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d6af1dd1-25a2-41cc-8785-8ba44c545cde tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:30.206981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d6af1dd1-25a2-41cc-8785-8ba44c545cde tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:30.213389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6af1dd1-25a2-41cc-8785-8ba44c545cde tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:30.213878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6af1dd1-25a2-41cc-8785-8ba44c545cde tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:30.214322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6af1dd1-25a2-41cc-8785-8ba44c545cde tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:30.215012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6af1dd1-25a2-41cc-8785-8ba44c545cde tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:30.215473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6af1dd1-25a2-41cc-8785-8ba44c545cde tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:30.215904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d6af1dd1-25a2-41cc-8785-8ba44c545cde tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:30.245800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d6af1dd1-25a2-41cc-8785-8ba44c545cde tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045282 Aug 24 10:27:30.246350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 795/1451] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:30 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:31.177164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:31.177718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:31.184190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:31.184674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:31.185108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:31.185798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:31.186251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:31.187177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:31.251517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:31.258161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-85cc957b-2786-4141-94a2-b7bd090559eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:31.259520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-81d0c636-5f07-4dc0-86c8-af860545f564 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086344 Aug 24 10:27:31.260113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 657/1452] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:31 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:31.260342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-85cc957b-2786-4141-94a2-b7bd090559eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:31.266626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85cc957b-2786-4141-94a2-b7bd090559eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:31.267110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85cc957b-2786-4141-94a2-b7bd090559eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:31.267550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85cc957b-2786-4141-94a2-b7bd090559eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:31.268272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85cc957b-2786-4141-94a2-b7bd090559eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:31.268727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85cc957b-2786-4141-94a2-b7bd090559eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:31.269153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85cc957b-2786-4141-94a2-b7bd090559eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:31.297731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-85cc957b-2786-4141-94a2-b7bd090559eb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043435 Aug 24 10:27:31.298285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 796/1453] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:31 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:32.281117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:32.281720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:32.288011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:32.288511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:32.288939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:32.289772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:32.290213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:32.290628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:32.309739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-22b7b420-a07d-4345-8eb4-6feaa7f87344 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:32.312491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-22b7b420-a07d-4345-8eb4-6feaa7f87344 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:32.318777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22b7b420-a07d-4345-8eb4-6feaa7f87344 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:32.319263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22b7b420-a07d-4345-8eb4-6feaa7f87344 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:32.319710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22b7b420-a07d-4345-8eb4-6feaa7f87344 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:32.320436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22b7b420-a07d-4345-8eb4-6feaa7f87344 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:32.320898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22b7b420-a07d-4345-8eb4-6feaa7f87344 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:32.321331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22b7b420-a07d-4345-8eb4-6feaa7f87344 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:32.351559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:32.351943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-22b7b420-a07d-4345-8eb4-6feaa7f87344 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046091 Aug 24 10:27:32.352557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 797/1454] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:32 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:32.359908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-386c74a4-67bc-4b4b-a606-b87720f06c4e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083191 Aug 24 10:27:32.360480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 658/1455] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:32 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:33.364311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-88221077-41bb-4965-a3ec-8ab1766b1356 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:33.366462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-88221077-41bb-4965-a3ec-8ab1766b1356 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:33.372755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88221077-41bb-4965-a3ec-8ab1766b1356 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:33.373235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88221077-41bb-4965-a3ec-8ab1766b1356 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:33.373676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88221077-41bb-4965-a3ec-8ab1766b1356 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:33.374361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88221077-41bb-4965-a3ec-8ab1766b1356 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:33.374816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88221077-41bb-4965-a3ec-8ab1766b1356 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:33.375239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88221077-41bb-4965-a3ec-8ab1766b1356 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:33.382522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:33.383174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:33.389440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:33.390044 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:33.390395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:33.391080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:33.391529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:33.391947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:33.403957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-88221077-41bb-4965-a3ec-8ab1766b1356 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043623 Aug 24 10:27:33.404574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 798/1456] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:33 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:33.452854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:33.460512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-375a5ba5-db8b-4b82-9758-e0ffb8829b60 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081739 Aug 24 10:27:33.461066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 659/1457] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:33 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:27:34.416589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6c3ba706-0006-4ae9-bc02-81ad8f124b3d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:34.418737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6c3ba706-0006-4ae9-bc02-81ad8f124b3d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:34.425570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c3ba706-0006-4ae9-bc02-81ad8f124b3d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:34.426066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c3ba706-0006-4ae9-bc02-81ad8f124b3d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:34.426499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c3ba706-0006-4ae9-bc02-81ad8f124b3d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:34.427177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c3ba706-0006-4ae9-bc02-81ad8f124b3d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:34.427625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c3ba706-0006-4ae9-bc02-81ad8f124b3d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:34.428060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c3ba706-0006-4ae9-bc02-81ad8f124b3d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:34.456901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6c3ba706-0006-4ae9-bc02-81ad8f124b3d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044278 Aug 24 10:27:34.457442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 799/1458] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:34 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:34.481474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:34.482091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:34.489318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:34.489797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:34.490234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:34.490924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:34.491377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:34.491801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:34.551796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:34.559686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-478bf28e-4d18-4033-afe0-0d15e33e6426 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082067 Aug 24 10:27:34.560287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 660/1459] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:34 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:35.469296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0432853e-7636-4729-abce-cd35abaddf65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:35.471412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0432853e-7636-4729-abce-cd35abaddf65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:35.477236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0432853e-7636-4729-abce-cd35abaddf65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:35.477709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0432853e-7636-4729-abce-cd35abaddf65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:35.478141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0432853e-7636-4729-abce-cd35abaddf65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:35.478820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0432853e-7636-4729-abce-cd35abaddf65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:35.479267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0432853e-7636-4729-abce-cd35abaddf65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:35.479686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0432853e-7636-4729-abce-cd35abaddf65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:35.508293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0432853e-7636-4729-abce-cd35abaddf65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043083 Aug 24 10:27:35.508754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 800/1460] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:35 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:35.581878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:35.582492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:35.589153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:35.589635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:35.590066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:35.590750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:35.591204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:35.591627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:35.609226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:35.658425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:35.666113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a9e2d7fd-169c-481d-bbb4-93f81ab87f07 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.088100 Aug 24 10:27:35.666884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 661/1461] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:35 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:36.521238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d9d55aa5-5aa3-4787-8058-295680737e3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:36.523388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d9d55aa5-5aa3-4787-8058-295680737e3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:36.530709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9d55aa5-5aa3-4787-8058-295680737e3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:36.531192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9d55aa5-5aa3-4787-8058-295680737e3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:36.531656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9d55aa5-5aa3-4787-8058-295680737e3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:36.532451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9d55aa5-5aa3-4787-8058-295680737e3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:36.532930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9d55aa5-5aa3-4787-8058-295680737e3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:36.533358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9d55aa5-5aa3-4787-8058-295680737e3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:36.561711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d9d55aa5-5aa3-4787-8058-295680737e3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044419 Aug 24 10:27:36.562283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 801/1462] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:36 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:36.686530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:36.687145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:36.693617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:36.694102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:36.694538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:36.695224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:36.695681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:36.696137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:36.756433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:36.765032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-28fd7b76-a470-4ac2-8a9f-58770a2c8a1e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082361 Aug 24 10:27:36.765601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 662/1463] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:36 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:37.573084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ff6d79e6-82f4-4dc9-b8e4-67ca6e55f2e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:37.575347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ff6d79e6-82f4-4dc9-b8e4-67ca6e55f2e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:37.581643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff6d79e6-82f4-4dc9-b8e4-67ca6e55f2e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:37.582122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff6d79e6-82f4-4dc9-b8e4-67ca6e55f2e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:37.582556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff6d79e6-82f4-4dc9-b8e4-67ca6e55f2e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:37.583233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff6d79e6-82f4-4dc9-b8e4-67ca6e55f2e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:37.583687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff6d79e6-82f4-4dc9-b8e4-67ca6e55f2e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:37.584132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ff6d79e6-82f4-4dc9-b8e4-67ca6e55f2e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:37.613013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ff6d79e6-82f4-4dc9-b8e4-67ca6e55f2e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043931 Aug 24 10:27:37.613533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 802/1464] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:37 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:37.786125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:37.786746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:37.793277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:37.793755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:37.794188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:37.794882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:37.795871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:37.796319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:37.856544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:37.864894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-60148953-c8df-47d0-99c8-304fa1f2733e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082713 Aug 24 10:27:37.865443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 663/1465] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:37 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:38.625793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7b33d7ef-a3b1-430f-b104-c9d99001c262 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:38.627830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7b33d7ef-a3b1-430f-b104-c9d99001c262 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:38.634290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b33d7ef-a3b1-430f-b104-c9d99001c262 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:38.634773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b33d7ef-a3b1-430f-b104-c9d99001c262 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:38.635216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b33d7ef-a3b1-430f-b104-c9d99001c262 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:38.635902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b33d7ef-a3b1-430f-b104-c9d99001c262 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:38.636382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b33d7ef-a3b1-430f-b104-c9d99001c262 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:38.636810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7b33d7ef-a3b1-430f-b104-c9d99001c262 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:38.665312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7b33d7ef-a3b1-430f-b104-c9d99001c262 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044186 Aug 24 10:27:38.665851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 803/1466] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:38 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:38.885427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:38.885972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:38.892363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:38.892847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:38.893287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:38.893973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:38.894442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:38.894856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:38.955060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:38.962756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2f9caebc-ef27-434f-8522-8e62d7c1c0be tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081384 Aug 24 10:27:38.963322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 664/1467] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:38 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:39.677758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d68ce423-b5b4-41b3-bba1-822c7f0d6b8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:39.679814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d68ce423-b5b4-41b3-bba1-822c7f0d6b8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:39.686788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d68ce423-b5b4-41b3-bba1-822c7f0d6b8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:39.687260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d68ce423-b5b4-41b3-bba1-822c7f0d6b8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:39.687692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d68ce423-b5b4-41b3-bba1-822c7f0d6b8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:39.688427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d68ce423-b5b4-41b3-bba1-822c7f0d6b8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:39.688876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d68ce423-b5b4-41b3-bba1-822c7f0d6b8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:39.689293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d68ce423-b5b4-41b3-bba1-822c7f0d6b8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:39.719691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d68ce423-b5b4-41b3-bba1-822c7f0d6b8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045886 Aug 24 10:27:39.720279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 804/1468] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:39 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:39.983438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:39.984092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:39.990540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:39.991023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:39.991459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:39.992177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:39.992670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:39.993146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:40.057817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:40.065906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-27c4cf14-88af-4829-9633-450e06b74801 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086956 Aug 24 10:27:40.066485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 665/1469] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:39 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:40.732025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-035fd8da-fcc0-4c18-bf1e-87d63c0aaecf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:40.734219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-035fd8da-fcc0-4c18-bf1e-87d63c0aaecf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:40.740573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-035fd8da-fcc0-4c18-bf1e-87d63c0aaecf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:40.741049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-035fd8da-fcc0-4c18-bf1e-87d63c0aaecf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:40.741482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-035fd8da-fcc0-4c18-bf1e-87d63c0aaecf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:40.742165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-035fd8da-fcc0-4c18-bf1e-87d63c0aaecf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:40.742613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-035fd8da-fcc0-4c18-bf1e-87d63c0aaecf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:40.743037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-035fd8da-fcc0-4c18-bf1e-87d63c0aaecf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:40.772686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-035fd8da-fcc0-4c18-bf1e-87d63c0aaecf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044606 Aug 24 10:27:40.773232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 805/1470] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:40 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:41.086870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:41.087443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:41.093849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:41.094332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:41.094771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:41.095461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:41.095916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:41.096371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:41.157377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:41.165010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e457c761-7c05-4e94-9485-6378b9460ef4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082106 Aug 24 10:27:41.165579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 666/1471] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:41 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:41.785447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e3a1c99b-dd21-4799-908c-9af50e16bf90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:41.787553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e3a1c99b-dd21-4799-908c-9af50e16bf90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:41.793899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3a1c99b-dd21-4799-908c-9af50e16bf90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:41.794378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3a1c99b-dd21-4799-908c-9af50e16bf90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:41.794814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3a1c99b-dd21-4799-908c-9af50e16bf90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:41.795501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3a1c99b-dd21-4799-908c-9af50e16bf90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:41.795949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3a1c99b-dd21-4799-908c-9af50e16bf90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:41.796397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3a1c99b-dd21-4799-908c-9af50e16bf90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:41.820106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:41.826049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e3a1c99b-dd21-4799-908c-9af50e16bf90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044912 Aug 24 10:27:41.827179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 806/1472] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:41 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:42.186852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:42.187466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:42.194027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:42.194513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:42.195018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:42.195793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:42.196117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:42.196543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:42.258467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:42.266842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ebd52ba8-301e-4338-98e0-13d0641e52f4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084414 Aug 24 10:27:42.267405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 667/1473] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:42 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:42.839021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-62dd70b3-0522-4606-be46-513ddbce2320 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:42.841196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-62dd70b3-0522-4606-be46-513ddbce2320 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:42.847981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62dd70b3-0522-4606-be46-513ddbce2320 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:42.848496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62dd70b3-0522-4606-be46-513ddbce2320 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:42.848937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62dd70b3-0522-4606-be46-513ddbce2320 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:42.849633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62dd70b3-0522-4606-be46-513ddbce2320 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:42.850095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62dd70b3-0522-4606-be46-513ddbce2320 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:42.850525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62dd70b3-0522-4606-be46-513ddbce2320 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:42.879590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-62dd70b3-0522-4606-be46-513ddbce2320 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044815 Aug 24 10:27:42.880151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 807/1474] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:42 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:43.287634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:43.288295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:43.294709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:43.295219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:43.295650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:43.296385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:43.296841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:43.297268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:43.360888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:43.368760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-616197c2-3d02-40cc-9075-a804008aae78 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085129 Aug 24 10:27:43.369326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 668/1475] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:43 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:43.892729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-891023e0-91b9-4cba-abdf-134060954c6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:43.894846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-891023e0-91b9-4cba-abdf-134060954c6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:43.901333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-891023e0-91b9-4cba-abdf-134060954c6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:43.901817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-891023e0-91b9-4cba-abdf-134060954c6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:43.902260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-891023e0-91b9-4cba-abdf-134060954c6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:43.902959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-891023e0-91b9-4cba-abdf-134060954c6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:43.903420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-891023e0-91b9-4cba-abdf-134060954c6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:43.903848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-891023e0-91b9-4cba-abdf-134060954c6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:43.932452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-891023e0-91b9-4cba-abdf-134060954c6b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044352 Aug 24 10:27:43.932995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 808/1476] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:43 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:44.390084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:44.390692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:44.397544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:44.398030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:44.398467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:44.399152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:44.399593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:44.400013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:44.464499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:44.473600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ea2b8a22-05a6-4143-870a-55eb3aba7e7f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087970 Aug 24 10:27:44.474174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 669/1477] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:44 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:44.944523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-18251145-d4e6-42b6-9bf5-a4410557d3cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:44.946645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-18251145-d4e6-42b6-9bf5-a4410557d3cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:44.953381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18251145-d4e6-42b6-9bf5-a4410557d3cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:44.953867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18251145-d4e6-42b6-9bf5-a4410557d3cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:44.954307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18251145-d4e6-42b6-9bf5-a4410557d3cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:44.955000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18251145-d4e6-42b6-9bf5-a4410557d3cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:44.955462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18251145-d4e6-42b6-9bf5-a4410557d3cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:44.955893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18251145-d4e6-42b6-9bf5-a4410557d3cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:44.986457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-18251145-d4e6-42b6-9bf5-a4410557d3cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045904 Aug 24 10:27:44.987020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 809/1478] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:44 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:45.494024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:45.494670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:45.501320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:45.501880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:45.502320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:45.503012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:45.503469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:45.503896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:45.564118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:45.571809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-84376700-3005-4176-a7e5-9b8105093cb9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081668 Aug 24 10:27:45.572402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 670/1479] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:45 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:45.999101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d3da7279-433a-4dcc-9b27-336dd5b45582 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:46.001398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d3da7279-433a-4dcc-9b27-336dd5b45582 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:46.008027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3da7279-433a-4dcc-9b27-336dd5b45582 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:46.008532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3da7279-433a-4dcc-9b27-336dd5b45582 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:46.008975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3da7279-433a-4dcc-9b27-336dd5b45582 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:46.009665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3da7279-433a-4dcc-9b27-336dd5b45582 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:46.010120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3da7279-433a-4dcc-9b27-336dd5b45582 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:46.010546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3da7279-433a-4dcc-9b27-336dd5b45582 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:46.039880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d3da7279-433a-4dcc-9b27-336dd5b45582 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045187 Aug 24 10:27:46.040401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 810/1480] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:45 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:46.593814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:46.594449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:46.601089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:46.601560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:46.601986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:46.602661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:46.603108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:46.603523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:46.663597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:46.671837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1e831300-bf91-404a-af70-0819c175589b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082403 Aug 24 10:27:46.672424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 671/1481] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:46 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:47.051045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-10611886-0e2b-4926-813a-072c5611cb8e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:47.053280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-10611886-0e2b-4926-813a-072c5611cb8e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:47.059693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10611886-0e2b-4926-813a-072c5611cb8e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:47.060317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10611886-0e2b-4926-813a-072c5611cb8e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:47.060767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10611886-0e2b-4926-813a-072c5611cb8e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:47.061457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10611886-0e2b-4926-813a-072c5611cb8e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:47.061915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10611886-0e2b-4926-813a-072c5611cb8e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:47.062343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10611886-0e2b-4926-813a-072c5611cb8e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:47.093402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-10611886-0e2b-4926-813a-072c5611cb8e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044578 Aug 24 10:27:47.093967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 811/1482] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:47 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:47.692389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:47.692999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:47.699553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:47.700329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:47.700797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:47.702117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:47.702576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:47.703001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:47.763639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:47.771793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-11986134-0fc2-4b35-9d4c-c01d00f77637 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083259 Aug 24 10:27:47.772397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 672/1483] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:47 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:48.105306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0c458699-cec6-4528-b9c7-f9b3f76577e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:48.107993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0c458699-cec6-4528-b9c7-f9b3f76577e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:48.114423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c458699-cec6-4528-b9c7-f9b3f76577e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:48.114916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c458699-cec6-4528-b9c7-f9b3f76577e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:48.115357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c458699-cec6-4528-b9c7-f9b3f76577e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:48.116098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c458699-cec6-4528-b9c7-f9b3f76577e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:48.116562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c458699-cec6-4528-b9c7-f9b3f76577e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:48.116993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0c458699-cec6-4528-b9c7-f9b3f76577e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:48.145450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0c458699-cec6-4528-b9c7-f9b3f76577e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044165 Aug 24 10:27:48.146002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 812/1484] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:48 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:48.792596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:48.793268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:48.800177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:48.800661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:48.801099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:48.801788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:48.802246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:48.802671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:48.866471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:48.874091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2a23e6a4-d38d-471a-a4c3-d3c2062d9932 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085426 Aug 24 10:27:48.874639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 673/1485] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:48 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:49.158242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c5e7b291-69c5-4d1b-90b2-a2948df8b85f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:49.160376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c5e7b291-69c5-4d1b-90b2-a2948df8b85f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:49.166842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e7b291-69c5-4d1b-90b2-a2948df8b85f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:49.167325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e7b291-69c5-4d1b-90b2-a2948df8b85f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:49.167760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e7b291-69c5-4d1b-90b2-a2948df8b85f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:49.168473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e7b291-69c5-4d1b-90b2-a2948df8b85f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:49.168926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e7b291-69c5-4d1b-90b2-a2948df8b85f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:49.169350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e7b291-69c5-4d1b-90b2-a2948df8b85f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:49.197920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c5e7b291-69c5-4d1b-90b2-a2948df8b85f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044209 Aug 24 10:27:49.198459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 813/1486] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:49 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:49.895264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:49.895872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:49.901979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:49.902454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:49.902883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:49.903568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:49.904019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:49.904468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:49.967703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:49.975297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b5193327-1547-49c0-a5b2-1beb691ad5ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083928 Aug 24 10:27:49.975857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 674/1487] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:49 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:50.210905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0d289a02-ff66-43de-91b2-39ba636fd063 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:50.213069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0d289a02-ff66-43de-91b2-39ba636fd063 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:50.219292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d289a02-ff66-43de-91b2-39ba636fd063 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:50.219802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d289a02-ff66-43de-91b2-39ba636fd063 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:50.220257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d289a02-ff66-43de-91b2-39ba636fd063 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:50.221003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d289a02-ff66-43de-91b2-39ba636fd063 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:50.221456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d289a02-ff66-43de-91b2-39ba636fd063 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:50.221881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d289a02-ff66-43de-91b2-39ba636fd063 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:50.251191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0d289a02-ff66-43de-91b2-39ba636fd063 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044349 Aug 24 10:27:50.251716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 814/1488] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:50 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:50.996382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:50.996997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:50.999288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=56 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:51.004460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:51.004949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:51.005386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:51.006077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:51.006534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:51.006963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:51.067713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:51.076562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-68668cd0-6261-4385-a013-2589a0bffc82 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083404 Aug 24 10:27:51.077143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 675/1489] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:50 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:51.264009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f1cc9a27-0f5e-4488-a494-37b0fab4663e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:51.266145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f1cc9a27-0f5e-4488-a494-37b0fab4663e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:51.272485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1cc9a27-0f5e-4488-a494-37b0fab4663e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:51.272971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1cc9a27-0f5e-4488-a494-37b0fab4663e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:51.273407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1cc9a27-0f5e-4488-a494-37b0fab4663e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:51.274098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1cc9a27-0f5e-4488-a494-37b0fab4663e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:51.274551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1cc9a27-0f5e-4488-a494-37b0fab4663e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:51.274977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f1cc9a27-0f5e-4488-a494-37b0fab4663e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:51.303482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f1cc9a27-0f5e-4488-a494-37b0fab4663e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044116 Aug 24 10:27:51.304023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 815/1490] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:51 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:52.097826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:52.098386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:52.105270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:52.105750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:52.106182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:52.106871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:52.107976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:52.108425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:52.168603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:52.177071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-666af45c-8fd0-4a7c-a648-743084831d5b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083311 Aug 24 10:27:52.177626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 676/1491] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:52 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:52.316068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7c6cb948-2f96-4b84-bfff-c09da2d61111 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:52.318201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7c6cb948-2f96-4b84-bfff-c09da2d61111 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:52.320615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=58 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:52.325979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c6cb948-2f96-4b84-bfff-c09da2d61111 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:52.326457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c6cb948-2f96-4b84-bfff-c09da2d61111 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:52.326895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c6cb948-2f96-4b84-bfff-c09da2d61111 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:52.327613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c6cb948-2f96-4b84-bfff-c09da2d61111 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:52.328094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c6cb948-2f96-4b84-bfff-c09da2d61111 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:52.328521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c6cb948-2f96-4b84-bfff-c09da2d61111 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:52.357531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7c6cb948-2f96-4b84-bfff-c09da2d61111 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044833 Aug 24 10:27:52.358022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 816/1492] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:52 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:53.197385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:53.198013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:53.204677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:53.205149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:53.205576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:53.206248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:53.206694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:53.207105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:53.270677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:53.278299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-98d1d975-84b2-4124-94ef-d1446edb0f5a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084763 Aug 24 10:27:53.278887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 677/1493] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:53 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:53.370708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b1eca64e-41d6-4876-9b73-d2c619f8a45a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:53.372857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b1eca64e-41d6-4876-9b73-d2c619f8a45a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:53.379100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1eca64e-41d6-4876-9b73-d2c619f8a45a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:53.379578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1eca64e-41d6-4876-9b73-d2c619f8a45a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:53.380010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1eca64e-41d6-4876-9b73-d2c619f8a45a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:53.380710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1eca64e-41d6-4876-9b73-d2c619f8a45a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:53.381163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1eca64e-41d6-4876-9b73-d2c619f8a45a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:53.381583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1eca64e-41d6-4876-9b73-d2c619f8a45a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:53.409906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b1eca64e-41d6-4876-9b73-d2c619f8a45a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043089 Aug 24 10:27:53.410432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 817/1494] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:54.298823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:54.299440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:54.305997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:54.306460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:54.306898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:54.307588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:54.308084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:54.308494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:54.312957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:54.369815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:54.377938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-44c695a2-4bc0-4272-bae1-b8d66ba683ff tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083033 Aug 24 10:27:54.378501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 678/1495] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:54 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:54.422593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-daac1df4-7222-42fe-a811-3c948accf002 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:54.424835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-daac1df4-7222-42fe-a811-3c948accf002 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:54.431150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daac1df4-7222-42fe-a811-3c948accf002 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:54.431636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daac1df4-7222-42fe-a811-3c948accf002 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:54.432105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daac1df4-7222-42fe-a811-3c948accf002 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:54.432800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daac1df4-7222-42fe-a811-3c948accf002 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:54.433260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daac1df4-7222-42fe-a811-3c948accf002 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:54.433738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daac1df4-7222-42fe-a811-3c948accf002 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:54.463064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-daac1df4-7222-42fe-a811-3c948accf002 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044874 Aug 24 10:27:54.463611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 818/1496] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:54 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:55.250033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:27:55.250200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:27:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:27:55.399008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:55.399659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:55.406670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:55.407157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:55.407597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:55.408325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:55.408785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:55.409211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:55.469891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:55.474892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f00138e0-972f-4838-a535-aa74dfdf572a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:55.477049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f00138e0-972f-4838-a535-aa74dfdf572a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:55.477700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-952dbb2a-2ccb-4672-b428-e2498143fa4c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082586 Aug 24 10:27:55.478268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 679/1497] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:55 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:55.483545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f00138e0-972f-4838-a535-aa74dfdf572a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:55.484028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f00138e0-972f-4838-a535-aa74dfdf572a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:55.484493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f00138e0-972f-4838-a535-aa74dfdf572a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:55.485486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f00138e0-972f-4838-a535-aa74dfdf572a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:55.485952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f00138e0-972f-4838-a535-aa74dfdf572a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:55.486378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f00138e0-972f-4838-a535-aa74dfdf572a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:55.523087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f00138e0-972f-4838-a535-aa74dfdf572a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051749 Aug 24 10:27:55.523723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 819/1498] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:55 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:56.497888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:56.498535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:56.504985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:56.505473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:56.505908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:56.506600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:56.507066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:56.507481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:56.535245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-372520d6-01f4-42e8-93ed-d4fa379dc8d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:56.537455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-372520d6-01f4-42e8-93ed-d4fa379dc8d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:56.543774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-372520d6-01f4-42e8-93ed-d4fa379dc8d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:56.544878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-372520d6-01f4-42e8-93ed-d4fa379dc8d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:56.545315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-372520d6-01f4-42e8-93ed-d4fa379dc8d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:56.546028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-372520d6-01f4-42e8-93ed-d4fa379dc8d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:56.546480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-372520d6-01f4-42e8-93ed-d4fa379dc8d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:56.546906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-372520d6-01f4-42e8-93ed-d4fa379dc8d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:56.569662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:56.576788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-372520d6-01f4-42e8-93ed-d4fa379dc8d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045445 Aug 24 10:27:56.577341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 820/1499] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:56 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:27:56.577494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-59edf92c-a68f-40f3-ac23-fe7fb0d86efb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083570 Aug 24 10:27:56.577942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 680/1500] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:56 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:57.590193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-30c0fa70-e898-46ff-82b2-e93f50f73f4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:57.591261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-30c0fa70-e898-46ff-82b2-e93f50f73f4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:57.597917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30c0fa70-e898-46ff-82b2-e93f50f73f4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:57.598071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:57.598533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30c0fa70-e898-46ff-82b2-e93f50f73f4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:57.598647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:57.598953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30c0fa70-e898-46ff-82b2-e93f50f73f4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:57.599639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30c0fa70-e898-46ff-82b2-e93f50f73f4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:57.600366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30c0fa70-e898-46ff-82b2-e93f50f73f4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:57.600769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30c0fa70-e898-46ff-82b2-e93f50f73f4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:57.607494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:57.607983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:57.608453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:57.609155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:57.609608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:57.610036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:57.629441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-30c0fa70-e898-46ff-82b2-e93f50f73f4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044353 Aug 24 10:27:57.629982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 821/1501] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:57 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:57.676151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:57.683929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7f1f22ec-90c9-4b3d-a77c-42dc81932df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.089707 Aug 24 10:27:57.684596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 681/1502] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:57 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:27:58.641670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c8e88c11-d162-4e7c-8332-572793c8318a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:58.643779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c8e88c11-d162-4e7c-8332-572793c8318a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:58.650216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8e88c11-d162-4e7c-8332-572793c8318a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:58.650696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8e88c11-d162-4e7c-8332-572793c8318a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:58.651132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8e88c11-d162-4e7c-8332-572793c8318a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:58.651825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8e88c11-d162-4e7c-8332-572793c8318a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:58.652306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8e88c11-d162-4e7c-8332-572793c8318a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:58.652731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8e88c11-d162-4e7c-8332-572793c8318a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:58.681764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c8e88c11-d162-4e7c-8332-572793c8318a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044085 Aug 24 10:27:58.682298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 822/1503] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:58 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:58.704236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:58.704883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:58.711329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:58.711816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:58.712295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:58.712982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:58.713435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:58.713859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:58.774199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:58.782476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-299b5598-3cf6-49e3-a572-54a82ad683ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082150 Aug 24 10:27:58.783252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 682/1504] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:58 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:27:59.694145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ee5bee79-d8b9-4741-a46d-45cb1668ad0e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:59.696392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ee5bee79-d8b9-4741-a46d-45cb1668ad0e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:59.702968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee5bee79-d8b9-4741-a46d-45cb1668ad0e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:59.703454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee5bee79-d8b9-4741-a46d-45cb1668ad0e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:59.703896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee5bee79-d8b9-4741-a46d-45cb1668ad0e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:59.704617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee5bee79-d8b9-4741-a46d-45cb1668ad0e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:59.705078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee5bee79-d8b9-4741-a46d-45cb1668ad0e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:59.705507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee5bee79-d8b9-4741-a46d-45cb1668ad0e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:59.721595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:27:59.735429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ee5bee79-d8b9-4741-a46d-45cb1668ad0e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045298 Aug 24 10:27:59.735975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 823/1505] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:27:59 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:27:59.803473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:27:59.804126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:27:59.811149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:59.811634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:59.812095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:59.812797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:27:59.813253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:27:59.813679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:27:59.874616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:27:59.882422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c75264ec-8807-4e29-828e-e0d2488b428e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082872 Aug 24 10:27:59.882983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 683/1506] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:27:59 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:00.747406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-25a0c195-a378-443c-9ab3-3cce6ade7683 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:00.749630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-25a0c195-a378-443c-9ab3-3cce6ade7683 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:00.755977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25a0c195-a378-443c-9ab3-3cce6ade7683 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:00.756490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25a0c195-a378-443c-9ab3-3cce6ade7683 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:00.756933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25a0c195-a378-443c-9ab3-3cce6ade7683 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:00.757624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25a0c195-a378-443c-9ab3-3cce6ade7683 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:00.758084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25a0c195-a378-443c-9ab3-3cce6ade7683 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:00.758537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-25a0c195-a378-443c-9ab3-3cce6ade7683 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:00.789009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-25a0c195-a378-443c-9ab3-3cce6ade7683 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045784 Aug 24 10:28:00.789564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 824/1507] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:00 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:00.902918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:00.903525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:00.909943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:00.910422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:00.910855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:00.911537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:00.911989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:00.912524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:00.973008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:00.981075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-44744d3b-697e-48fe-856b-c145e162ecfe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082058 Aug 24 10:28:00.981666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 684/1508] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:00 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:01.801258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-66321288-bef2-4da0-aefc-bf7aa23c23ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:01.803389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-66321288-bef2-4da0-aefc-bf7aa23c23ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:01.809823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66321288-bef2-4da0-aefc-bf7aa23c23ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:01.810313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66321288-bef2-4da0-aefc-bf7aa23c23ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:01.810756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66321288-bef2-4da0-aefc-bf7aa23c23ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:01.811454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66321288-bef2-4da0-aefc-bf7aa23c23ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:01.811911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66321288-bef2-4da0-aefc-bf7aa23c23ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:01.812560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66321288-bef2-4da0-aefc-bf7aa23c23ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:01.844312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-66321288-bef2-4da0-aefc-bf7aa23c23ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046909 Aug 24 10:28:01.844874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 825/1509] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:01 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:02.000662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:02.001393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:02.007781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:02.008300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:02.008733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:02.009414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:02.009861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:02.010277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:02.073825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:02.082231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-05d6df03-3a86-44e6-b9fe-a6abe486e106 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085442 Aug 24 10:28:02.082779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 685/1510] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:01 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:02.857105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-db15423c-2779-4c3a-bba0-f35d8d231b3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:02.859238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-db15423c-2779-4c3a-bba0-f35d8d231b3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:02.865617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db15423c-2779-4c3a-bba0-f35d8d231b3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:02.866096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db15423c-2779-4c3a-bba0-f35d8d231b3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:02.866523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db15423c-2779-4c3a-bba0-f35d8d231b3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:02.867203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db15423c-2779-4c3a-bba0-f35d8d231b3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:02.867649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db15423c-2779-4c3a-bba0-f35d8d231b3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:02.868089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db15423c-2779-4c3a-bba0-f35d8d231b3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:02.896720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-db15423c-2779-4c3a-bba0-f35d8d231b3a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044052 Aug 24 10:28:02.897252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 826/1511] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:02 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:03.103294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:03.103862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:03.110841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:03.111320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:03.111752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:03.112467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:03.112918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:03.113343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:03.174057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:03.181780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-50b4d23d-6eeb-47bd-8211-c0d276d54bf6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082513 Aug 24 10:28:03.182339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 686/1512] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:03 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:03.909209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2b577ceb-a6a3-4656-b4f4-abd33dbc28cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:03.911343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2b577ceb-a6a3-4656-b4f4-abd33dbc28cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:03.917920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b577ceb-a6a3-4656-b4f4-abd33dbc28cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:03.918413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b577ceb-a6a3-4656-b4f4-abd33dbc28cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:03.918855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b577ceb-a6a3-4656-b4f4-abd33dbc28cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:03.919557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b577ceb-a6a3-4656-b4f4-abd33dbc28cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:03.920704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b577ceb-a6a3-4656-b4f4-abd33dbc28cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:03.921131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2b577ceb-a6a3-4656-b4f4-abd33dbc28cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:03.951140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2b577ceb-a6a3-4656-b4f4-abd33dbc28cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045841 Aug 24 10:28:03.951677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 827/1513] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:03 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:04.202722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:04.203325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:04.209813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:04.210289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:04.210714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:04.211395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:04.211868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:04.212314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:04.274968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:04.283476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-747a7936-1088-4453-b513-81e06be84ade tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085523 Aug 24 10:28:04.284067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 687/1514] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:04.963213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a1b48b5d-8659-4708-863f-24dccf686530 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:04.965312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a1b48b5d-8659-4708-863f-24dccf686530 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:04.971620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1b48b5d-8659-4708-863f-24dccf686530 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:04.972131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1b48b5d-8659-4708-863f-24dccf686530 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:04.972567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1b48b5d-8659-4708-863f-24dccf686530 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:04.973247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1b48b5d-8659-4708-863f-24dccf686530 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:04.973695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1b48b5d-8659-4708-863f-24dccf686530 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:04.974141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a1b48b5d-8659-4708-863f-24dccf686530 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:05.004086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a1b48b5d-8659-4708-863f-24dccf686530 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045125 Aug 24 10:28:05.004633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 828/1515] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:04 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:05.304147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:05.304833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:05.311963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:05.312495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:05.312933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:05.313626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:05.314081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:05.314510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:05.375066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:05.382808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6c68295c-b88c-4ebd-81f1-630de0c6ec2c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082978 Aug 24 10:28:05.383503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 688/1516] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:06.016399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5f3b7cf5-11e6-452e-94d1-2acbc210ea31 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:06.018548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5f3b7cf5-11e6-452e-94d1-2acbc210ea31 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:06.024745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3b7cf5-11e6-452e-94d1-2acbc210ea31 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:06.025174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3b7cf5-11e6-452e-94d1-2acbc210ea31 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:06.025614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3b7cf5-11e6-452e-94d1-2acbc210ea31 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:06.026302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3b7cf5-11e6-452e-94d1-2acbc210ea31 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:06.026757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3b7cf5-11e6-452e-94d1-2acbc210ea31 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:06.027182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f3b7cf5-11e6-452e-94d1-2acbc210ea31 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:06.056692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5f3b7cf5-11e6-452e-94d1-2acbc210ea31 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044325 Aug 24 10:28:06.057229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 829/1517] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:06 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:06.402668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:06.403309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:06.410377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:06.410861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:06.411298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:06.412125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:06.412512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:06.412922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:06.478753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:06.487612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-862f3745-e433-462e-83c7-360e678ddee1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.088696 Aug 24 10:28:06.488460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 689/1518] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:06 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:06.661802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:06.662298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:06.668092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:06.668584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:06.669023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:06.669706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:06.670158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:06.670579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:06.694900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:28:06.781633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e730462d-a86a-440b-a4a3-ea3caeae2027 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "DELETE /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 204 len: 0 microversion: 2.20 time: 0.123348 Aug 24 10:28:06.782605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 830/1519] 10.222.0.86 () {60 vars in 1280 bytes} [Wed Aug 24 10:28:06 2022] DELETE /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 24 10:28:06.796420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:06.797046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:06.803375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:06.803993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:06.804275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:06.804976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:06.805460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:06.806421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:06.933843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:06.943425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:06.944835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:06.945982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-721bf55e-2464-492f-94a1-3b22281d3d17 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153708 Aug 24 10:28:06.946535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 690/1520] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:28:06.982108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:28:06.982310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:28:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:28:07.069506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eba4bbce-d706-4773-ae9e-d040674c349f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:07.071661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eba4bbce-d706-4773-ae9e-d040674c349f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:07.079172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba4bbce-d706-4773-ae9e-d040674c349f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:07.079661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba4bbce-d706-4773-ae9e-d040674c349f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:07.080149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba4bbce-d706-4773-ae9e-d040674c349f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:07.080898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba4bbce-d706-4773-ae9e-d040674c349f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:07.081364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba4bbce-d706-4773-ae9e-d040674c349f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:07.081822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eba4bbce-d706-4773-ae9e-d040674c349f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:07.112026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eba4bbce-d706-4773-ae9e-d040674c349f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046886 Aug 24 10:28:07.112601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 831/1521] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:07 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:07.508839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:07.509486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:07.516014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:07.516522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:07.516953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:07.517636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:07.518089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:07.518512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:07.580722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:07.588377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d08e2828-d687-4c76-9954-e66961aa82a2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083455 Aug 24 10:28:07.588937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 691/1522] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:07 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:07.970400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:07.971017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:07.976949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:07.977435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:07.977876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:07.978569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:07.979025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:07.979452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:08.111640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:08.120543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:08.122040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:08.123185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3d688a12-27ef-4099-9d1c-a70c46321ac7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157380 Aug 24 10:28:08.123730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 832/1523] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:08.124355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3cfcd979-8d32-48be-a141-c95238d040c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:08.126523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3cfcd979-8d32-48be-a141-c95238d040c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:08.132574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cfcd979-8d32-48be-a141-c95238d040c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:08.133056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cfcd979-8d32-48be-a141-c95238d040c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:08.133492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cfcd979-8d32-48be-a141-c95238d040c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:08.134174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cfcd979-8d32-48be-a141-c95238d040c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:08.134627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cfcd979-8d32-48be-a141-c95238d040c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:08.135051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cfcd979-8d32-48be-a141-c95238d040c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:08.164511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3cfcd979-8d32-48be-a141-c95238d040c0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044083 Aug 24 10:28:08.165051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 692/1524] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:08 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:08.609294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:08.609906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:08.615906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:08.616481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:08.616929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:08.617619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:08.618078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:08.618561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:08.679857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:08.688404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-43440436-6090-470d-8144-5730141e5e7c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082868 Aug 24 10:28:08.688968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 833/1525] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:08 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:09.145351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:09.145945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:09.152386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:09.152865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:09.153297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:09.153976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:09.154433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:09.155353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:09.177121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4edfd258-da21-4d75-91ee-3ca39df95d73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:09.179383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4edfd258-da21-4d75-91ee-3ca39df95d73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:09.185737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4edfd258-da21-4d75-91ee-3ca39df95d73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:09.186227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4edfd258-da21-4d75-91ee-3ca39df95d73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:09.186674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4edfd258-da21-4d75-91ee-3ca39df95d73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:09.187430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4edfd258-da21-4d75-91ee-3ca39df95d73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:09.187889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4edfd258-da21-4d75-91ee-3ca39df95d73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:09.188348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4edfd258-da21-4d75-91ee-3ca39df95d73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:09.219068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4edfd258-da21-4d75-91ee-3ca39df95d73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045804 Aug 24 10:28:09.219626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 834/1526] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:09 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:09.285291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:09.293929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:09.296139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:09.297296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fe2cf2e8-ad25-4437-bdcf-2ba25a0f5cfa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155894 Aug 24 10:28:09.297847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 693/1527] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:09 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:09.709119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:09.709652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:09.715415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:09.715875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:09.716338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:09.717016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:09.717460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:09.717878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:09.777696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:09.786045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5c625415-d6ca-4e54-bfc1-aa9f02d4c331 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080895 Aug 24 10:28:09.786599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 835/1528] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:09 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:10.231275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-13425e6f-8059-43bf-ae28-5b5511ee23d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:10.234462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-13425e6f-8059-43bf-ae28-5b5511ee23d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:10.241307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13425e6f-8059-43bf-ae28-5b5511ee23d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:10.241781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13425e6f-8059-43bf-ae28-5b5511ee23d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:10.242210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13425e6f-8059-43bf-ae28-5b5511ee23d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:10.242898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13425e6f-8059-43bf-ae28-5b5511ee23d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:10.243345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13425e6f-8059-43bf-ae28-5b5511ee23d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:10.243761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13425e6f-8059-43bf-ae28-5b5511ee23d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:10.275278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:10.281208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-13425e6f-8059-43bf-ae28-5b5511ee23d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054014 Aug 24 10:28:10.281758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 694/1529] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:10 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:10.319171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:10.319787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:10.327192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:10.327680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:10.328146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:10.328844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:10.329301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:10.329732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:10.458974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:10.468320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:10.469831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:10.470988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fb3b4348-78be-494a-a463-282a06444181 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155763 Aug 24 10:28:10.471542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 836/1530] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:10 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:28:10.654052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:28:10.654272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:28:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:28:10.806939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:10.807552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:10.814246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:10.814731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:10.815166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:10.815847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:10.816329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:10.816749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:10.879382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:10.887090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cfcb0e26-3238-4e6e-a746-abfe870cc10f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084098 Aug 24 10:28:10.887648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 695/1531] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:10 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:11.293519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fc2d9409-41ee-4679-8923-be463abe8d09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:11.296462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fc2d9409-41ee-4679-8923-be463abe8d09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:11.302411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc2d9409-41ee-4679-8923-be463abe8d09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:11.302885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc2d9409-41ee-4679-8923-be463abe8d09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:11.303315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc2d9409-41ee-4679-8923-be463abe8d09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:11.303990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc2d9409-41ee-4679-8923-be463abe8d09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:11.304472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc2d9409-41ee-4679-8923-be463abe8d09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:11.304891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc2d9409-41ee-4679-8923-be463abe8d09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:11.333319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fc2d9409-41ee-4679-8923-be463abe8d09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043809 Aug 24 10:28:11.333935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 837/1532] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:11 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:11.493680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:11.494291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:11.500683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:11.501162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:11.501596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:11.502284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:11.502739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:11.503162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:11.628767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:11.637237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:11.638736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:11.640406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e012f318-340a-462b-95f6-5857ebb7e8dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.150660 Aug 24 10:28:11.640962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 696/1533] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:11 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:11.918988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:11.919530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:11.925433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:11.925921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:11.926358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:11.927040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:11.927494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:11.927923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:11.945553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:11.990382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:11.997920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-764ffa7b-b5de-46ce-9cce-af58f57fb8b5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082851 Aug 24 10:28:11.998970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 838/1534] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:11 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:12.101297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:28:12.101466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:28:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:28:12.345704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7baa2f37-56af-473a-a554-4803c8b05b24 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:12.347889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7baa2f37-56af-473a-a554-4803c8b05b24 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:12.353791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7baa2f37-56af-473a-a554-4803c8b05b24 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:12.354264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7baa2f37-56af-473a-a554-4803c8b05b24 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:12.354689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7baa2f37-56af-473a-a554-4803c8b05b24 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:12.355368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7baa2f37-56af-473a-a554-4803c8b05b24 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:12.355815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7baa2f37-56af-473a-a554-4803c8b05b24 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:12.356260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7baa2f37-56af-473a-a554-4803c8b05b24 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:12.385781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7baa2f37-56af-473a-a554-4803c8b05b24 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044056 Aug 24 10:28:12.386316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 697/1535] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:12 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:12.664614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:12.665282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:12.672500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:12.672984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:12.673446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:12.674168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:12.674622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:12.675046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:12.807006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:12.815505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:12.817036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:12.818474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5dd5fbcf-90de-452f-af94-c55fb2b1d51a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157835 Aug 24 10:28:12.819036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 839/1536] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:12 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:13.018744 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:13.019401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:13.026520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:13.027029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:13.027492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:13.028232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:13.028713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:13.029159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:13.095921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:13.103628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ba95abee-caa3-41fd-a469-27fd4386ce30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.088798 Aug 24 10:28:13.104200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 698/1537] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:13 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:13.399936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f38cc19c-c204-497d-af0f-b64fe42636ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:13.402200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f38cc19c-c204-497d-af0f-b64fe42636ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:13.408801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f38cc19c-c204-497d-af0f-b64fe42636ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:13.409289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f38cc19c-c204-497d-af0f-b64fe42636ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:13.409731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f38cc19c-c204-497d-af0f-b64fe42636ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:13.410421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f38cc19c-c204-497d-af0f-b64fe42636ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:13.410878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f38cc19c-c204-497d-af0f-b64fe42636ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:13.411307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f38cc19c-c204-497d-af0f-b64fe42636ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:13.441164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f38cc19c-c204-497d-af0f-b64fe42636ec tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046298 Aug 24 10:28:13.441701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 840/1538] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:13 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:13.841922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:13.842528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:13.848569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:13.849146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:13.849586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:13.850278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:13.850731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:13.851278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:13.977751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:13.987327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:13.989046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:13.990235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5e1d35ef-a59f-4c27-98ad-98377928ce0a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152308 Aug 24 10:28:13.990798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 699/1539] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:14.125483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:14.126023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:14.132369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:14.132849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:14.133283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:14.133962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:14.134440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:14.134863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:14.199328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:14.207033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-93616a06-2be0-4b41-87b7-4c840230895b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086122 Aug 24 10:28:14.207581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 841/1540] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:14 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:14.453740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dd053539-9a06-4e3f-ad52-2006ef6293ba tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:14.456072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dd053539-9a06-4e3f-ad52-2006ef6293ba tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:14.462767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd053539-9a06-4e3f-ad52-2006ef6293ba tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:14.463287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd053539-9a06-4e3f-ad52-2006ef6293ba tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:14.463758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd053539-9a06-4e3f-ad52-2006ef6293ba tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:14.464504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd053539-9a06-4e3f-ad52-2006ef6293ba tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:14.465001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd053539-9a06-4e3f-ad52-2006ef6293ba tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:14.465462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd053539-9a06-4e3f-ad52-2006ef6293ba tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:14.496280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dd053539-9a06-4e3f-ad52-2006ef6293ba tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046582 Aug 24 10:28:14.496868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 700/1541] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:14 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:15.017092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:15.018026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:15.025003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:15.025426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:15.025863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:15.026548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:15.026998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:15.027423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:15.153391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:15.162175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:15.163671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:15.164863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1e3e7a11-07ed-4aca-ae16-f7527494634b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153727 Aug 24 10:28:15.165403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 842/1542] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:15.227447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:15.228069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:15.233950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:15.234425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:15.234855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:15.235539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:15.235988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:15.236435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:15.297411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:15.305164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-04d14bdb-5515-47fa-afca-7e1a18168241 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081638 Aug 24 10:28:15.305724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 701/1543] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:15 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:15.509150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eccd4924-9925-47dc-905e-de8aa4307f04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:15.511260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eccd4924-9925-47dc-905e-de8aa4307f04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:15.517165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eccd4924-9925-47dc-905e-de8aa4307f04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:15.517639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eccd4924-9925-47dc-905e-de8aa4307f04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:15.518068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eccd4924-9925-47dc-905e-de8aa4307f04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:15.518749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eccd4924-9925-47dc-905e-de8aa4307f04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:15.519194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eccd4924-9925-47dc-905e-de8aa4307f04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:15.519618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eccd4924-9925-47dc-905e-de8aa4307f04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:15.548838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eccd4924-9925-47dc-905e-de8aa4307f04 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043511 Aug 24 10:28:15.549307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 843/1544] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:15 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:16.187769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:16.188430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:16.194906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:16.195415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:16.195873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:16.196617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:16.197099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:16.197550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:16.329241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:16.329602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:16.330224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:16.337434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:16.337991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:16.338594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:16.338881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:16.339685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:16.340402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:16.340525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:16.340849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:16.341602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1b15f209-659b-4b60-a162-3ef351bc2ce4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157806 Aug 24 10:28:16.342156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 702/1545] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:16 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:16.409948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:16.417830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-78925f8d-17f7-421c-9471-db00c016bb99 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.093560 Aug 24 10:28:16.418391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 844/1546] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:16 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:16.561096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c9b8f465-20e1-4a96-b3db-058af278b927 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:16.563800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c9b8f465-20e1-4a96-b3db-058af278b927 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:16.570153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9b8f465-20e1-4a96-b3db-058af278b927 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:16.570669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9b8f465-20e1-4a96-b3db-058af278b927 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:16.571134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9b8f465-20e1-4a96-b3db-058af278b927 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:16.571852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9b8f465-20e1-4a96-b3db-058af278b927 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:16.572485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9b8f465-20e1-4a96-b3db-058af278b927 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:16.572940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c9b8f465-20e1-4a96-b3db-058af278b927 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:16.602067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c9b8f465-20e1-4a96-b3db-058af278b927 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045124 Aug 24 10:28:16.602613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 703/1547] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:16 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:17.365994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:17.366610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:17.373047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:17.373531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:17.373976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:17.374666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:17.375127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:17.375555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:17.438085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:17.438702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:17.445122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:17.445602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:17.446035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:17.446731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:17.447181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:17.447600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:17.507337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:17.511307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:17.516171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:17.517683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:17.518842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-15a09253-26e1-4788-b18f-71c5c9fd82d9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156772 Aug 24 10:28:17.519394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 845/1548] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:17.519539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a61ebe8a-e705-4e82-950a-100ff2fa109c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085089 Aug 24 10:28:17.520135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 704/1549] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:17 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:17.613880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6e89ca03-3f0f-403c-a6b7-8b5e368a4b91 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:17.616029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6e89ca03-3f0f-403c-a6b7-8b5e368a4b91 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:17.621922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e89ca03-3f0f-403c-a6b7-8b5e368a4b91 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:17.622406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e89ca03-3f0f-403c-a6b7-8b5e368a4b91 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:17.622848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e89ca03-3f0f-403c-a6b7-8b5e368a4b91 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:17.623539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e89ca03-3f0f-403c-a6b7-8b5e368a4b91 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:17.623996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e89ca03-3f0f-403c-a6b7-8b5e368a4b91 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:17.624459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e89ca03-3f0f-403c-a6b7-8b5e368a4b91 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:17.652946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6e89ca03-3f0f-403c-a6b7-8b5e368a4b91 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043054 Aug 24 10:28:17.653490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 846/1550] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:17 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:18.541212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:18.541818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:18.541920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:18.542441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:18.548340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:18.548816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:18.548951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:18.549245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:18.549449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:18.549919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:18.550045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:18.550363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:18.550607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:18.550779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:18.551081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:18.551513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:18.612814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:18.620563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-66e8c4a0-f8cc-49b5-94d2-753c30624019 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083743 Aug 24 10:28:18.621135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 705/1551] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:18 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:18.665859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-83624657-a969-42e9-8613-2e7aceb51651 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:18.668021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-83624657-a969-42e9-8613-2e7aceb51651 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:18.673926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-83624657-a969-42e9-8613-2e7aceb51651 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:18.674406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-83624657-a969-42e9-8613-2e7aceb51651 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:18.674837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-83624657-a969-42e9-8613-2e7aceb51651 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:18.675525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-83624657-a969-42e9-8613-2e7aceb51651 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:18.675978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-83624657-a969-42e9-8613-2e7aceb51651 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:18.676422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-83624657-a969-42e9-8613-2e7aceb51651 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:18.681007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:18.690159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:18.691672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:18.692871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3fde117b-36fc-4335-930f-0776b5cc02a6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154967 Aug 24 10:28:18.693482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 847/1552] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:18 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:28:18.708174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-83624657-a969-42e9-8613-2e7aceb51651 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046135 Aug 24 10:28:18.709060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 706/1553] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:18 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:19.641422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:19.642036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:19.648352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:19.648829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:19.649261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:19.649940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:19.650389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:19.650810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:19.712505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:19.716309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:19.716928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:19.720435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-69fdd83f-5757-4e3a-b44a-a7d4fd956b34 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082767 Aug 24 10:28:19.721514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 848/1554] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:19 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:19.730247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:19.730247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:19.730247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:19.730247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:19.730646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:19.730646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:19.730803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7099c480-790e-4a8a-b4e4-a63e870650b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:19.730803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7099c480-790e-4a8a-b4e4-a63e870650b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:19.734068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7099c480-790e-4a8a-b4e4-a63e870650b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:19.734544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7099c480-790e-4a8a-b4e4-a63e870650b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:19.734992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7099c480-790e-4a8a-b4e4-a63e870650b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:19.735689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7099c480-790e-4a8a-b4e4-a63e870650b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:19.736862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7099c480-790e-4a8a-b4e4-a63e870650b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:19.737299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7099c480-790e-4a8a-b4e4-a63e870650b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:19.767752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7099c480-790e-4a8a-b4e4-a63e870650b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045099 Aug 24 10:28:19.769703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 849/1555] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:19 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:19.859229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:19.868073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:19.869587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:19.870754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ec66233d-d61e-46fa-b44e-69054d67b47f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158348 Aug 24 10:28:19.871335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 707/1556] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:28:20.741361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:20.741978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:20.748412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:20.748893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:20.749333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:20.750021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:20.750476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:20.750901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:20.780146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-67ca2ea1-968d-41aa-8b18-ba7536fdd135 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:20.782285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-67ca2ea1-968d-41aa-8b18-ba7536fdd135 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:20.789116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67ca2ea1-968d-41aa-8b18-ba7536fdd135 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:20.789593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67ca2ea1-968d-41aa-8b18-ba7536fdd135 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:20.790016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67ca2ea1-968d-41aa-8b18-ba7536fdd135 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:20.790704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67ca2ea1-968d-41aa-8b18-ba7536fdd135 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:20.791149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67ca2ea1-968d-41aa-8b18-ba7536fdd135 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:20.791597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67ca2ea1-968d-41aa-8b18-ba7536fdd135 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:20.818812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:20.822600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-67ca2ea1-968d-41aa-8b18-ba7536fdd135 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046382 Aug 24 10:28:20.823166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 708/1557] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:20 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:20.826596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eee10d2c-ab46-4332-b388-11aa5ba78ef2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.089352 Aug 24 10:28:20.827136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 850/1558] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:20 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:20.893378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:20.893982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:20.900005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:20.900516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:20.900949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:20.901630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:20.902077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:20.902495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:21.029477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:21.038513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:21.040023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:21.041198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e5e217cf-d6b8-49ba-adb6-5a8c493f1683 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151799 Aug 24 10:28:21.041749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 709/1559] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:20 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:21.835643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9e13b9e2-227c-4d7e-96ff-bd5b02c21f42 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:21.837793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9e13b9e2-227c-4d7e-96ff-bd5b02c21f42 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:21.844161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e13b9e2-227c-4d7e-96ff-bd5b02c21f42 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:21.844639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e13b9e2-227c-4d7e-96ff-bd5b02c21f42 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:21.845072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e13b9e2-227c-4d7e-96ff-bd5b02c21f42 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:21.845870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e13b9e2-227c-4d7e-96ff-bd5b02c21f42 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:21.846319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e13b9e2-227c-4d7e-96ff-bd5b02c21f42 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:21.846538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:21.846961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e13b9e2-227c-4d7e-96ff-bd5b02c21f42 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:21.847156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:21.849505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=70 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:21.855069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:21.855595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:21.856083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:21.856863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:21.857364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:21.857835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:21.876123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9e13b9e2-227c-4d7e-96ff-bd5b02c21f42 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044370 Aug 24 10:28:21.876902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 851/1560] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:21 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:21.920792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:21.928574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b94ea17d-6cf3-4116-b88f-d7351ac76c11 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085747 Aug 24 10:28:21.929169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 710/1561] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:21 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:22.063691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:22.064247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:22.070959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:22.071439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:22.071880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:22.072611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:22.073062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:22.073738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:22.199947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:22.208626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:22.210122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:22.211282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-af6fa123-8304-49d0-bd66-bb650adcc6a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151571 Aug 24 10:28:22.211828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 852/1562] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:22 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:22.890024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c260fab3-21c1-44d1-9902-3c2444df086f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:22.892185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c260fab3-21c1-44d1-9902-3c2444df086f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:22.899256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c260fab3-21c1-44d1-9902-3c2444df086f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:22.899739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c260fab3-21c1-44d1-9902-3c2444df086f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:22.900213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c260fab3-21c1-44d1-9902-3c2444df086f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:22.900900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c260fab3-21c1-44d1-9902-3c2444df086f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:22.901352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c260fab3-21c1-44d1-9902-3c2444df086f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:22.901775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c260fab3-21c1-44d1-9902-3c2444df086f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:22.919054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:22.932860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c260fab3-21c1-44d1-9902-3c2444df086f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047583 Aug 24 10:28:22.933414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 711/1563] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:22 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:22.950918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:22.951524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:22.953872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=79,nova_api:UPDATE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:22.959442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:22.959925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:22.960393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:22.961090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:22.961582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:22.962006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:23.023341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:23.031038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-861919aa-bfa3-40ef-81c4-61dd4a2b6840 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084389 Aug 24 10:28:23.031589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 853/1564] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:22 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:23.233838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:23.234449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:23.240630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:23.241107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:23.241539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:23.242218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:23.242665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:23.243121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:23.373037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:23.382186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:23.383698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:23.384897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-567a611d-4dbe-40ac-bb11-67a76d2ad543 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155011 Aug 24 10:28:23.385454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 712/1565] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:23 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:23.946518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8c2495d8-4769-4ef9-a454-f5caf2ba70ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:23.948085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8c2495d8-4769-4ef9-a454-f5caf2ba70ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:23.954342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c2495d8-4769-4ef9-a454-f5caf2ba70ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:23.954926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c2495d8-4769-4ef9-a454-f5caf2ba70ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:23.955245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c2495d8-4769-4ef9-a454-f5caf2ba70ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:23.955935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c2495d8-4769-4ef9-a454-f5caf2ba70ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:23.957162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c2495d8-4769-4ef9-a454-f5caf2ba70ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:23.957586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c2495d8-4769-4ef9-a454-f5caf2ba70ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:23.986912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8c2495d8-4769-4ef9-a454-f5caf2ba70ae tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044991 Aug 24 10:28:23.987476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 854/1566] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:23 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:24.052621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:24.053193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:24.059657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:24.060175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:24.060614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:24.061306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:24.061760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:24.062186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:24.123049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:24.131285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a16fc38c-8f1b-4a1e-8218-9aed246cf565 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082857 Aug 24 10:28:24.131841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 713/1567] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:24 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:24.408464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:24.409030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:24.415255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:24.415731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:24.416204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:24.416940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:24.417433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:24.417859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:24.421883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:24.551075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:24.559663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:24.561188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:24.562327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cb2b44d9-064e-4920-984f-bd1e7a88c958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157874 Aug 24 10:28:24.562871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 855/1568] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:24 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:24.998833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f3c9b709-c376-4442-8da8-4a862c04b27c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:25.001146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f3c9b709-c376-4442-8da8-4a862c04b27c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:25.007129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3c9b709-c376-4442-8da8-4a862c04b27c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:25.007612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3c9b709-c376-4442-8da8-4a862c04b27c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:25.008058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3c9b709-c376-4442-8da8-4a862c04b27c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:25.008750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3c9b709-c376-4442-8da8-4a862c04b27c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:25.009236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3c9b709-c376-4442-8da8-4a862c04b27c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:25.009659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3c9b709-c376-4442-8da8-4a862c04b27c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:25.040574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f3c9b709-c376-4442-8da8-4a862c04b27c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045548 Aug 24 10:28:25.041053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 714/1569] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:24 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:25.152407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:25.153028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:25.160115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:25.160604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:25.161049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:25.161736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:25.162197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:25.162628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:25.225598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:25.233181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a0b458f2-2bbe-4b4b-be39-7c7d4126d0d8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084717 Aug 24 10:28:25.233738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 856/1570] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:25 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:25.586322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:25.586928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:25.593356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:25.593835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:25.594266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:25.594949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:25.595401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:25.595822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:25.724960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:25.733559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:25.735049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:25.736243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-688b12a8-13a9-46cb-a804-7a6d36289f01 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153822 Aug 24 10:28:25.736790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 715/1571] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:25 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:26.052881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-697fb552-4c6c-4e5d-b640-669ab705ec35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:26.055679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-697fb552-4c6c-4e5d-b640-669ab705ec35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:26.061755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-697fb552-4c6c-4e5d-b640-669ab705ec35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:26.062243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-697fb552-4c6c-4e5d-b640-669ab705ec35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:26.062687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-697fb552-4c6c-4e5d-b640-669ab705ec35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:26.063384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-697fb552-4c6c-4e5d-b640-669ab705ec35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:26.063842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-697fb552-4c6c-4e5d-b640-669ab705ec35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:26.064302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-697fb552-4c6c-4e5d-b640-669ab705ec35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:26.092905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-697fb552-4c6c-4e5d-b640-669ab705ec35 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044048 Aug 24 10:28:26.093454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 857/1572] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:26 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:26.252835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:26.253466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:26.259506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:26.260017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:26.260520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:26.261235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:26.261725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:26.262161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:26.327072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:26.335362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b86a7474-052b-41c7-8d71-c6b26a6a6078 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086525 Aug 24 10:28:26.335949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 716/1573] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:26 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:26.759936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:26.760660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:26.767257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:26.767739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:26.768210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:26.768908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:26.769395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:26.769829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:26.897290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:26.906541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:26.908021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:26.909188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f2a33c00-daf9-4599-9350-440558bbfc02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153951 Aug 24 10:28:26.909732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 858/1574] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:26 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:27.105442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2917116a-2ce1-4194-b567-2ff881f329e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:27.107595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2917116a-2ce1-4194-b567-2ff881f329e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:27.114040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2917116a-2ce1-4194-b567-2ff881f329e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:27.114523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2917116a-2ce1-4194-b567-2ff881f329e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:27.114958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2917116a-2ce1-4194-b567-2ff881f329e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:27.115651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2917116a-2ce1-4194-b567-2ff881f329e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:27.116126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2917116a-2ce1-4194-b567-2ff881f329e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:27.116553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2917116a-2ce1-4194-b567-2ff881f329e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:27.146068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2917116a-2ce1-4194-b567-2ff881f329e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044688 Aug 24 10:28:27.146616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 717/1575] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:27 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:27.356368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:27.356924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:27.362821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:27.363301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:27.363830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:27.364539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:27.364994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:27.365419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:27.428438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:27.436731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6e95770a-fa71-4045-91c0-2f1ca297e355 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084291 Aug 24 10:28:27.437284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 859/1576] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:27 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:27.931846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:27.932477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:27.938423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:27.938901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:27.939339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:27.940026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:27.940496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:27.940917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:28.066347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:28.075233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:28.076802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:28.077960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1d31ceee-c438-488a-8106-570f04e40d59 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.150146 Aug 24 10:28:28.078514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 718/1577] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:27 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:28.157801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dd3ae22e-ccdc-4cd6-93c8-1d95c527ead8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:28.159907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dd3ae22e-ccdc-4cd6-93c8-1d95c527ead8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:28.166950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd3ae22e-ccdc-4cd6-93c8-1d95c527ead8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:28.167418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd3ae22e-ccdc-4cd6-93c8-1d95c527ead8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:28.167848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd3ae22e-ccdc-4cd6-93c8-1d95c527ead8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:28.168603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd3ae22e-ccdc-4cd6-93c8-1d95c527ead8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:28.169053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd3ae22e-ccdc-4cd6-93c8-1d95c527ead8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:28.169473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dd3ae22e-ccdc-4cd6-93c8-1d95c527ead8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:28.198294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dd3ae22e-ccdc-4cd6-93c8-1d95c527ead8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044142 Aug 24 10:28:28.198835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 860/1578] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:28 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:28.458221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:28.458835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:28.465905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:28.466385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:28.466820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:28.467498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:28.467953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:28.468405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:28.528656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:28.536303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-905e31fd-21ca-4b27-9846-4a9514088762 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082052 Aug 24 10:28:28.537453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 719/1579] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:28 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:29.101181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:29.101707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:29.108126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:29.108617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:29.109064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:29.109749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:29.110206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:29.110636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:29.210478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3deb05b7-c96d-4566-972d-a2bc8dabdd4b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:29.212743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3deb05b7-c96d-4566-972d-a2bc8dabdd4b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:29.218875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3deb05b7-c96d-4566-972d-a2bc8dabdd4b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:29.219366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3deb05b7-c96d-4566-972d-a2bc8dabdd4b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:29.219805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3deb05b7-c96d-4566-972d-a2bc8dabdd4b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:29.220567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3deb05b7-c96d-4566-972d-a2bc8dabdd4b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:29.221034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3deb05b7-c96d-4566-972d-a2bc8dabdd4b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:29.221472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3deb05b7-c96d-4566-972d-a2bc8dabdd4b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:29.239669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:29.250173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:29.251903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:29.252181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3deb05b7-c96d-4566-972d-a2bc8dabdd4b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045662 Aug 24 10:28:29.252756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 720/1580] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:29 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:29.253186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e99cf00c-54f3-4930-acec-3cd60964f2f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156015 Aug 24 10:28:29.253760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 861/1581] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:29.557672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:29.558271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:29.564723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:29.565207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:29.565646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:29.566334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:29.566787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:29.567215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:29.630481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:29.638837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-52075e7c-8f3d-4f75-805b-8e3ec9fcebc4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085628 Aug 24 10:28:29.639408 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 721/1582] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:29 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:30.264124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f861a458-c370-4e69-8103-b85dd22b8cf8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:30.266265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f861a458-c370-4e69-8103-b85dd22b8cf8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:30.272667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f861a458-c370-4e69-8103-b85dd22b8cf8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:30.273151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f861a458-c370-4e69-8103-b85dd22b8cf8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:30.273605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f861a458-c370-4e69-8103-b85dd22b8cf8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:30.274368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f861a458-c370-4e69-8103-b85dd22b8cf8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:30.274747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f861a458-c370-4e69-8103-b85dd22b8cf8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:30.275657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f861a458-c370-4e69-8103-b85dd22b8cf8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:30.277965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:30.278651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:30.285288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:30.285875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:30.286382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:30.287147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:30.287674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:30.288207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:30.308508 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f861a458-c370-4e69-8103-b85dd22b8cf8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048368 Aug 24 10:28:30.313251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 862/1583] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:30 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:30.427083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:30.436027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:30.437808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:30.438974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b7479f50-af7e-4f0f-a6fd-3085edd10418 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.165984 Aug 24 10:28:30.439524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 722/1584] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:28:30.659191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:30.659806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:30.665762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:30.666291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:30.666709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:30.667397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:30.667852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:30.668309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:30.757793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:30.766536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-da2503f1-5e71-4f57-b2fa-f9a0dcaa7c5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.111240 Aug 24 10:28:30.767143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 863/1585] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:30 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:31.320885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4181f56b-18a8-4f2c-8f0a-9d2c8ff50ced tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:31.323798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4181f56b-18a8-4f2c-8f0a-9d2c8ff50ced tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:31.330228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4181f56b-18a8-4f2c-8f0a-9d2c8ff50ced tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:31.330711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4181f56b-18a8-4f2c-8f0a-9d2c8ff50ced tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:31.331146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4181f56b-18a8-4f2c-8f0a-9d2c8ff50ced tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:31.331830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4181f56b-18a8-4f2c-8f0a-9d2c8ff50ced tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:31.332311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4181f56b-18a8-4f2c-8f0a-9d2c8ff50ced tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:31.332737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4181f56b-18a8-4f2c-8f0a-9d2c8ff50ced tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:31.361364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4181f56b-18a8-4f2c-8f0a-9d2c8ff50ced tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044448 Aug 24 10:28:31.361908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 723/1586] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:31 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:31.463095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:31.463714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:31.470224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:31.470705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:31.471142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:31.471829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:31.472308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:31.472734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:31.602381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:31.610964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:31.612505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:31.613695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e9cd4075-bf08-40ea-8808-08b415303ac6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155310 Aug 24 10:28:31.614235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 864/1587] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:31 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:31.787563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:31.788230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:31.794295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:31.794775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:31.795207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:31.795894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:31.796381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:31.796807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:31.856901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:31.864971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4811e50a-7396-46e6-8749-958544638deb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081079 Aug 24 10:28:31.865560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 724/1588] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:31 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:32.373822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7834b9f6-6265-4013-8522-11ac6abff6c2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:32.375964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7834b9f6-6265-4013-8522-11ac6abff6c2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:32.382035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7834b9f6-6265-4013-8522-11ac6abff6c2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:32.382524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7834b9f6-6265-4013-8522-11ac6abff6c2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:32.382964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7834b9f6-6265-4013-8522-11ac6abff6c2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:32.383685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7834b9f6-6265-4013-8522-11ac6abff6c2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:32.384177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7834b9f6-6265-4013-8522-11ac6abff6c2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:32.384605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7834b9f6-6265-4013-8522-11ac6abff6c2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:32.413181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7834b9f6-6265-4013-8522-11ac6abff6c2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043403 Aug 24 10:28:32.413722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 865/1589] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:32 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:32.636898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:32.637439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:32.643949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:32.644467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:32.644903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:32.645590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:32.646042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:32.646533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:32.781870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:32.790635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:32.792154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:32.793310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9eecc0b7-10f8-46cc-8556-6ab44b207aab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.160434 Aug 24 10:28:32.793864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 725/1590] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:32 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:32.886361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:32.887029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:32.893320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:32.893806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:32.894241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:32.895021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:32.895474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:32.895901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:32.958273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:32.966488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ffbc98e8-161c-4af0-a426-d7c5faf42e30 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084557 Aug 24 10:28:32.967042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 866/1591] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:32 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:33.426487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-955cb34c-1a39-4afc-a56f-adb5c1ab9b2e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:33.428691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-955cb34c-1a39-4afc-a56f-adb5c1ab9b2e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:33.435177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955cb34c-1a39-4afc-a56f-adb5c1ab9b2e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:33.435670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955cb34c-1a39-4afc-a56f-adb5c1ab9b2e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:33.436139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955cb34c-1a39-4afc-a56f-adb5c1ab9b2e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:33.436839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955cb34c-1a39-4afc-a56f-adb5c1ab9b2e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:33.437294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955cb34c-1a39-4afc-a56f-adb5c1ab9b2e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:33.437720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955cb34c-1a39-4afc-a56f-adb5c1ab9b2e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:33.466801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-955cb34c-1a39-4afc-a56f-adb5c1ab9b2e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044379 Aug 24 10:28:33.467345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 726/1592] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:33 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:33.815665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:33.816313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:33.822387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:33.823122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:33.823559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:33.824282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:33.824734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:33.825158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:33.951614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:33.960351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:33.961856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:33.963526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ecebd325-9844-495f-8a88-668c5bfe8fb6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151764 Aug 24 10:28:33.964088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 867/1593] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:33.986557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:33.987803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:33.994469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:33.995010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:33.995440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:33.996148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:33.996627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:33.997126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:34.058920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:34.067200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-14780085-a2e0-41b8-9b2d-04ee449069d9 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084456 Aug 24 10:28:34.067766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 727/1594] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:33 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:34.479368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5112a664-8a14-492c-860e-6de95f911214 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:34.481592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5112a664-8a14-492c-860e-6de95f911214 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:34.487574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5112a664-8a14-492c-860e-6de95f911214 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:34.488086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5112a664-8a14-492c-860e-6de95f911214 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:34.488536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5112a664-8a14-492c-860e-6de95f911214 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:34.489234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5112a664-8a14-492c-860e-6de95f911214 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:34.489692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5112a664-8a14-492c-860e-6de95f911214 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:34.490123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5112a664-8a14-492c-860e-6de95f911214 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:34.519348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5112a664-8a14-492c-860e-6de95f911214 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044159 Aug 24 10:28:34.519902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 868/1595] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:34 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:34.986946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:34.987636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:34.995032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:34.995578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:34.996017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:34.996739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:34.997250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:34.997690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:35.001808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:35.087849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:35.088549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:35.094968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:35.096082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:35.096082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:35.096883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:35.097406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:35.097931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:35.128909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:35.138070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:35.139681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:35.140879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ea20ac24-340a-4e4f-bb46-0c3774f1d1da tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158004 Aug 24 10:28:35.141482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 728/1596] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:34 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:35.163125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:35.171503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-07998f89-63ab-4ef9-b9c8-c26294e08f66 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087489 Aug 24 10:28:35.172106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 869/1597] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:35 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:35.531390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-98316c76-6c10-4e0b-9b4c-cc0dea620d4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:35.533571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-98316c76-6c10-4e0b-9b4c-cc0dea620d4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:35.539404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98316c76-6c10-4e0b-9b4c-cc0dea620d4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:35.539879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98316c76-6c10-4e0b-9b4c-cc0dea620d4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:35.541011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98316c76-6c10-4e0b-9b4c-cc0dea620d4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:35.541697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98316c76-6c10-4e0b-9b4c-cc0dea620d4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:35.542143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98316c76-6c10-4e0b-9b4c-cc0dea620d4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:35.542565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98316c76-6c10-4e0b-9b4c-cc0dea620d4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:35.570607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-98316c76-6c10-4e0b-9b4c-cc0dea620d4f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043209 Aug 24 10:28:35.571153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 729/1598] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:35 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:36.163057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:36.163665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:36.169589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:36.170072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:36.170501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:36.171174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:36.171622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:36.172068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:36.192813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:36.193455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:36.201032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:36.201472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:36.201911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:36.202608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:36.203098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:36.203546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:36.271929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:36.279622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-21dd75c1-a7d2-433e-8a83-daf66090c532 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.090882 Aug 24 10:28:36.280342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 730/1599] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:36 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:36.299600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:36.308275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:36.310305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:36.311455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4299606c-105c-46d9-8dd4-5ef458170bc1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152618 Aug 24 10:28:36.312001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 870/1600] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:36 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:36.583271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-95b342e4-172d-41a0-bd8a-5537d1c34435 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:36.585457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-95b342e4-172d-41a0-bd8a-5537d1c34435 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:36.591401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95b342e4-172d-41a0-bd8a-5537d1c34435 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:36.591883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95b342e4-172d-41a0-bd8a-5537d1c34435 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:36.592356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95b342e4-172d-41a0-bd8a-5537d1c34435 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:36.593045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95b342e4-172d-41a0-bd8a-5537d1c34435 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:36.593500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95b342e4-172d-41a0-bd8a-5537d1c34435 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:36.593925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95b342e4-172d-41a0-bd8a-5537d1c34435 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:36.623539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-95b342e4-172d-41a0-bd8a-5537d1c34435 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044269 Aug 24 10:28:36.624102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 731/1601] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:36 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:37.300145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:37.300794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:37.307212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:37.307695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:37.308170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:37.308861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:37.309318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:37.309745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:37.334726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:37.335335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:37.342341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:37.342827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:37.343260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:37.343951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:37.344426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:37.344847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:37.370783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:37.374258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:37.379440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-872fd2fd-ac2b-4859-b0d4-fdded475b4e7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083126 Aug 24 10:28:37.380003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 871/1602] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:37 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:37.473682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:37.482320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:37.483834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:37.485015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b0e1d008-76c7-48ff-a434-5ad500defa51 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154274 Aug 24 10:28:37.485576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 732/1603] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:28:37.636842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-33fb72ba-c308-4da3-ae60-2ecd6979246c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:37.639598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-33fb72ba-c308-4da3-ae60-2ecd6979246c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:37.645446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33fb72ba-c308-4da3-ae60-2ecd6979246c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:37.645929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33fb72ba-c308-4da3-ae60-2ecd6979246c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:37.646359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33fb72ba-c308-4da3-ae60-2ecd6979246c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:37.647042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33fb72ba-c308-4da3-ae60-2ecd6979246c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:37.647491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33fb72ba-c308-4da3-ae60-2ecd6979246c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:37.647911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33fb72ba-c308-4da3-ae60-2ecd6979246c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:37.675987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-33fb72ba-c308-4da3-ae60-2ecd6979246c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043310 Aug 24 10:28:37.676560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 872/1604] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:37 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:38.401575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:38.402190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:38.408631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:38.409110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:38.409545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:38.410229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:38.410679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:38.411100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:38.472381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:38.480000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8e1c6875-b4d4-432f-9fd5-c8bc1d65dd1f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082382 Aug 24 10:28:38.480570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 733/1605] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:38 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:38.508213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:38.508921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:38.515843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:38.516355 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:38.516790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:38.517493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:38.517943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:38.518363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:38.647032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:38.655563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:38.657080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:38.658257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8d1224c3-10ea-4431-9640-9577094835dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153949 Aug 24 10:28:38.658796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 873/1606] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:38 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:38.688683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4bbb0f0d-36f5-44f2-b26f-5c80e8689ca6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:38.690861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4bbb0f0d-36f5-44f2-b26f-5c80e8689ca6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:38.696841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bbb0f0d-36f5-44f2-b26f-5c80e8689ca6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:38.697325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bbb0f0d-36f5-44f2-b26f-5c80e8689ca6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:38.698022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bbb0f0d-36f5-44f2-b26f-5c80e8689ca6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:38.698713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bbb0f0d-36f5-44f2-b26f-5c80e8689ca6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:38.699167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bbb0f0d-36f5-44f2-b26f-5c80e8689ca6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:38.699594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bbb0f0d-36f5-44f2-b26f-5c80e8689ca6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:38.727684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4bbb0f0d-36f5-44f2-b26f-5c80e8689ca6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043841 Aug 24 10:28:38.728250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 734/1607] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:38 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:39.499854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:39.500474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:39.506396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:39.506870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:39.507303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:39.507975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:39.508456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:39.509012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:39.570156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:39.578448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-554e9947-44bd-4965-bb12-588f6a2f1f0d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082518 Aug 24 10:28:39.578991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 874/1608] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:39 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:39.681906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:39.682511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:39.688924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:39.689402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:39.689837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:39.690554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:39.691002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:39.691428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:39.740742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c711c082-94c7-4bc9-8929-ca16884bbd9e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:39.742906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c711c082-94c7-4bc9-8929-ca16884bbd9e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:39.749707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c711c082-94c7-4bc9-8929-ca16884bbd9e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:39.750203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c711c082-94c7-4bc9-8929-ca16884bbd9e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:39.750647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c711c082-94c7-4bc9-8929-ca16884bbd9e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:39.751358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c711c082-94c7-4bc9-8929-ca16884bbd9e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:39.751817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c711c082-94c7-4bc9-8929-ca16884bbd9e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:39.752329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c711c082-94c7-4bc9-8929-ca16884bbd9e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:39.783534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c711c082-94c7-4bc9-8929-ca16884bbd9e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046805 Aug 24 10:28:39.784426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 875/1609] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:39 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:39.830483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:39.839197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:39.840716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:39.841867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-44fdcf5a-b4b2-495d-ac32-a45e494da0b5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.164038 Aug 24 10:28:39.842417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 735/1610] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:39 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:40.598534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:40.599148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:40.605506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:40.605993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:40.606435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:40.607677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:40.608165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:40.608599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:40.667902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:40.676183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0faf8f23-fb1e-4bf3-942b-f92d82b8a676 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081469 Aug 24 10:28:40.676737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 876/1611] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:40 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:40.795647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-094978c3-78e2-4e8d-a55f-56bc271d88c3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:40.797799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-094978c3-78e2-4e8d-a55f-56bc271d88c3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:40.804725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-094978c3-78e2-4e8d-a55f-56bc271d88c3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:40.805207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-094978c3-78e2-4e8d-a55f-56bc271d88c3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:40.805635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-094978c3-78e2-4e8d-a55f-56bc271d88c3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:40.806624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-094978c3-78e2-4e8d-a55f-56bc271d88c3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:40.807176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-094978c3-78e2-4e8d-a55f-56bc271d88c3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:40.807596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-094978c3-78e2-4e8d-a55f-56bc271d88c3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:40.837664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-094978c3-78e2-4e8d-a55f-56bc271d88c3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046722 Aug 24 10:28:40.838208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 736/1612] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:40 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:40.864470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:40.865009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:40.870860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:40.871337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:40.871772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:40.872479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:40.872935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:40.873355 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:41.003295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:41.011756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:41.013769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:41.014908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dfc616f9-0ff4-4694-925b-35bb0ac8e99e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154474 Aug 24 10:28:41.015447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 877/1613] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:41.696296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:41.696911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:41.703555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:41.704036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:41.704492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:41.705176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:41.705625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:41.706050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:41.774563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:41.783010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e6dfa9f1-7182-4bb5-be0e-c0c3547898bf tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.090591 Aug 24 10:28:41.783576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 737/1614] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:41 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:41.850521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1bcd92c4-4b3f-4c6b-aef5-dd7715721a2f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:41.852673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1bcd92c4-4b3f-4c6b-aef5-dd7715721a2f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:41.858984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bcd92c4-4b3f-4c6b-aef5-dd7715721a2f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:41.859469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bcd92c4-4b3f-4c6b-aef5-dd7715721a2f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:41.859907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bcd92c4-4b3f-4c6b-aef5-dd7715721a2f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:41.860622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bcd92c4-4b3f-4c6b-aef5-dd7715721a2f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:41.861081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bcd92c4-4b3f-4c6b-aef5-dd7715721a2f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:41.861509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1bcd92c4-4b3f-4c6b-aef5-dd7715721a2f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:41.890239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1bcd92c4-4b3f-4c6b-aef5-dd7715721a2f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043681 Aug 24 10:28:41.890774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 878/1615] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:41 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:42.037621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:42.038138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:42.044512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:42.044994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:42.045429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:42.046119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:42.046566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:42.046985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:42.173672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:42.182260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:42.183761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:42.185001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a379aebf-c100-419b-b081-e94481ac99b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151402 Aug 24 10:28:42.185556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 738/1616] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:42 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:42.802747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:42.803362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:42.809274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:42.812164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:42.812624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:42.813318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:42.813785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:42.814235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:42.879020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:42.886643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e14d817e-4b82-4cdd-bfd9-63613a56515c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087759 Aug 24 10:28:42.887190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 879/1617] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:42 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:42.901596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1e97254d-dce9-497f-b07f-6ade2b195abe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:42.903725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1e97254d-dce9-497f-b07f-6ade2b195abe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:42.909760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e97254d-dce9-497f-b07f-6ade2b195abe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:42.910240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e97254d-dce9-497f-b07f-6ade2b195abe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:42.910683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e97254d-dce9-497f-b07f-6ade2b195abe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:42.911374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e97254d-dce9-497f-b07f-6ade2b195abe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:42.911829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e97254d-dce9-497f-b07f-6ade2b195abe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:42.912287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1e97254d-dce9-497f-b07f-6ade2b195abe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:42.943667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1e97254d-dce9-497f-b07f-6ade2b195abe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045925 Aug 24 10:28:42.944255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 739/1618] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:42 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:43.207985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:43.208618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:43.214468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:43.214954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:43.215385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:43.216089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:43.216539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:43.216961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:43.343500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:43.352193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:43.353686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:43.354831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-93cb7e5d-60d2-41e4-8d62-966fe688904e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.150760 Aug 24 10:28:43.355369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 880/1619] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:43.908726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:43.909427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:43.915820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:43.916331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:43.916780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:43.917489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:43.917929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:43.918353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:43.956816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-97702da6-d60f-45c4-8c96-9390b2a3a837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:43.959034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-97702da6-d60f-45c4-8c96-9390b2a3a837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:43.965484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97702da6-d60f-45c4-8c96-9390b2a3a837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:43.966630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97702da6-d60f-45c4-8c96-9390b2a3a837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:43.967048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97702da6-d60f-45c4-8c96-9390b2a3a837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:43.967759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97702da6-d60f-45c4-8c96-9390b2a3a837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:43.968252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97702da6-d60f-45c4-8c96-9390b2a3a837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:43.968679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97702da6-d60f-45c4-8c96-9390b2a3a837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:43.983678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:43.992927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4a5335dd-e13f-4eed-aafe-817087a178e5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.088123 Aug 24 10:28:43.993558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 740/1620] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:43 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:44.001422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-97702da6-d60f-45c4-8c96-9390b2a3a837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048636 Aug 24 10:28:44.002189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 881/1621] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:43 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:44.378137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:44.378742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:44.384723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:44.385203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:44.385640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:44.386319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:44.386768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:44.387190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:44.512784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:44.521557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:44.523070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:44.524288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0c18f85a-a220-4099-bc8e-493308d45753 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.150057 Aug 24 10:28:44.524844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 741/1622] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:45.013608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-43abd72a-efea-4af9-b2db-4d1ec2b4e714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:45.015039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:45.015652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:45.015771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-43abd72a-efea-4af9-b2db-4d1ec2b4e714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:45.022375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:45.022553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43abd72a-efea-4af9-b2db-4d1ec2b4e714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:45.022785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:45.022907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43abd72a-efea-4af9-b2db-4d1ec2b4e714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:45.023215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:45.023316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43abd72a-efea-4af9-b2db-4d1ec2b4e714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:45.023908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:45.024008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43abd72a-efea-4af9-b2db-4d1ec2b4e714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:45.024388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:45.024492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43abd72a-efea-4af9-b2db-4d1ec2b4e714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:45.024824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43abd72a-efea-4af9-b2db-4d1ec2b4e714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:45.024970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:45.054418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-43abd72a-efea-4af9-b2db-4d1ec2b4e714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045393 Aug 24 10:28:45.054980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 882/1623] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:45 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:45.086029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:45.093889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-174d4c14-f976-450a-a6d9-090d70a50408 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083245 Aug 24 10:28:45.094450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 742/1624] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:45 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:45.547601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:45.548165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:45.554101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:45.554586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:45.555024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:45.555710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:45.556195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:45.556622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:45.683925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:45.692444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:45.693932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:45.695073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-66f0e6a3-0309-406b-8e52-be1d1a4c3e72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151526 Aug 24 10:28:45.695613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 883/1625] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:45 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:46.066684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-146b7794-e581-4e09-8c58-da6365ce4471 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:46.068913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-146b7794-e581-4e09-8c58-da6365ce4471 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:46.076386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-146b7794-e581-4e09-8c58-da6365ce4471 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:46.076867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-146b7794-e581-4e09-8c58-da6365ce4471 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:46.077299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-146b7794-e581-4e09-8c58-da6365ce4471 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:46.078001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-146b7794-e581-4e09-8c58-da6365ce4471 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:46.078448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-146b7794-e581-4e09-8c58-da6365ce4471 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:46.078868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-146b7794-e581-4e09-8c58-da6365ce4471 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:46.107911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-146b7794-e581-4e09-8c58-da6365ce4471 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045125 Aug 24 10:28:46.108497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 743/1626] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:46 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:46.114971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:46.115584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:46.122635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:46.123115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:46.123565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:46.124272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:46.124723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:46.125145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:46.186388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:46.194063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-917712b0-1670-4823-870e-b05dca3c5740 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083687 Aug 24 10:28:46.194605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 884/1627] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:46 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:46.718853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:46.719459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:46.725420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:46.725895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:46.726328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:46.727006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:46.727457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:46.727875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:46.858083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:46.861481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:46.868004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:46.869538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:46.870691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d1b96a59-712a-48b0-9b3a-e8beacac96b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156282 Aug 24 10:28:46.871249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 744/1628] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:46 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:47.119447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eb5ce9ec-e8bb-41aa-ac8e-85934bc7d942 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:47.121535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eb5ce9ec-e8bb-41aa-ac8e-85934bc7d942 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:47.127864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb5ce9ec-e8bb-41aa-ac8e-85934bc7d942 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:47.128381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb5ce9ec-e8bb-41aa-ac8e-85934bc7d942 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:47.128821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb5ce9ec-e8bb-41aa-ac8e-85934bc7d942 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:47.129513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb5ce9ec-e8bb-41aa-ac8e-85934bc7d942 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:47.129994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb5ce9ec-e8bb-41aa-ac8e-85934bc7d942 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:47.130418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eb5ce9ec-e8bb-41aa-ac8e-85934bc7d942 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:47.159965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eb5ce9ec-e8bb-41aa-ac8e-85934bc7d942 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044533 Aug 24 10:28:47.160530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 885/1629] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:47 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:47.214542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:47.215146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:47.222241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:47.222715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:47.223141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:47.223814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:47.224294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:47.224710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:47.285394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:47.293011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0e5e4663-b3f1-4eed-8ed7-a5ef074bde5d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082283 Aug 24 10:28:47.293588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 745/1630] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:47 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:47.893478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:47.894131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:47.900203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:47.900700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:47.901142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:47.901833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:47.902292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:47.902721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:48.028588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:48.037360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:48.038878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:48.040068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-795ee091-86a7-4e0e-8f6f-92b9b655a19c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.150606 Aug 24 10:28:48.040618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 886/1631] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:48.172884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b95a66bd-2809-458b-a600-6bd18548821b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:48.175077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b95a66bd-2809-458b-a600-6bd18548821b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:48.181137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b95a66bd-2809-458b-a600-6bd18548821b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:48.181624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b95a66bd-2809-458b-a600-6bd18548821b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:48.182065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b95a66bd-2809-458b-a600-6bd18548821b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:48.182757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b95a66bd-2809-458b-a600-6bd18548821b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:48.183221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b95a66bd-2809-458b-a600-6bd18548821b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:48.183660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b95a66bd-2809-458b-a600-6bd18548821b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:48.214127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b95a66bd-2809-458b-a600-6bd18548821b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045846 Aug 24 10:28:48.214687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 746/1632] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:48 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:48.312594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:48.313118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:48.319521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:48.319998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:48.320459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:48.321139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:48.321588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:48.322011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:48.385389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:48.393573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5cb9b050-d5fe-4341-9827-072566c4195a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084865 Aug 24 10:28:48.394026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 887/1633] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:48 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:49.066188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:49.066750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:49.073721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:49.074204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:49.075306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:49.076010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:49.076485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:49.076912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:49.202675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:49.211693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:49.213222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:49.214375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f3ae0af3-fdb0-4f15-aceb-72d3deb47d11 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152246 Aug 24 10:28:49.214956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 747/1634] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:49.226651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-04b72d6a-0ec0-412c-913b-48f690545fd4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:49.228811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-04b72d6a-0ec0-412c-913b-48f690545fd4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:49.234680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b72d6a-0ec0-412c-913b-48f690545fd4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:49.235162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b72d6a-0ec0-412c-913b-48f690545fd4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:49.235605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b72d6a-0ec0-412c-913b-48f690545fd4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:49.236387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b72d6a-0ec0-412c-913b-48f690545fd4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:49.236845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b72d6a-0ec0-412c-913b-48f690545fd4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:49.237283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b72d6a-0ec0-412c-913b-48f690545fd4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:49.253232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:49.267057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-04b72d6a-0ec0-412c-913b-48f690545fd4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044349 Aug 24 10:28:49.267601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 888/1635] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:49 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:49.414908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:49.415441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:49.421569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:49.422047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:49.422485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:49.423514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:49.423979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:49.424438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:49.487852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:49.495430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c433b7ec-ed4f-4384-831a-139c34074e29 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084484 Aug 24 10:28:49.495994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 748/1636] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:49 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:50.237813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:50.238425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:50.244885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:50.245375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:50.245816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:50.247138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:50.247604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:50.248034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:50.279870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3097db27-94fc-4312-9234-c83f1576c238 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:50.282025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3097db27-94fc-4312-9234-c83f1576c238 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:50.287993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3097db27-94fc-4312-9234-c83f1576c238 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:50.288527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3097db27-94fc-4312-9234-c83f1576c238 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:50.288957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3097db27-94fc-4312-9234-c83f1576c238 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:50.289649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3097db27-94fc-4312-9234-c83f1576c238 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:50.290096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3097db27-94fc-4312-9234-c83f1576c238 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:50.290521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3097db27-94fc-4312-9234-c83f1576c238 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:50.322557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3097db27-94fc-4312-9234-c83f1576c238 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046875 Aug 24 10:28:50.323127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 749/1637] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:50 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:50.375226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:50.383833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:50.385390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:50.386551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-780563c1-2d27-4145-8c35-2f8407c2cf85 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152695 Aug 24 10:28:50.387115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 889/1638] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:50 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:50.516575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:50.517180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:50.523364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:50.523838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:50.524295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:50.524978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:50.525420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:50.525837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:50.586429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:50.594567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e13c54f2-e382-474f-9957-15019b60b5ab tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081783 Aug 24 10:28:50.595123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 750/1639] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:50 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:51.334649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4482128f-e6ac-4a1c-9536-641b97c0dd57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:51.336820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4482128f-e6ac-4a1c-9536-641b97c0dd57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:51.343197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4482128f-e6ac-4a1c-9536-641b97c0dd57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:51.343675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4482128f-e6ac-4a1c-9536-641b97c0dd57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:51.344137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4482128f-e6ac-4a1c-9536-641b97c0dd57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:51.344819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4482128f-e6ac-4a1c-9536-641b97c0dd57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:51.345270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4482128f-e6ac-4a1c-9536-641b97c0dd57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:51.345690 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4482128f-e6ac-4a1c-9536-641b97c0dd57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:51.374748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4482128f-e6ac-4a1c-9536-641b97c0dd57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044014 Aug 24 10:28:51.375289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 890/1640] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:51 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:51.409959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:51.410562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:51.416688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:51.417163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:51.417597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:51.418799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:51.419253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:51.419675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:51.544284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:51.552899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:51.554412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:51.555566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-389963d0-0b56-49c8-8ca9-4ed10c62a95f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.149635 Aug 24 10:28:51.556140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 751/1641] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:51.615314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:51.615923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:51.622332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:51.622811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:51.623243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:51.623928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:51.624398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:51.624897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:51.683760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:51.692037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c5173067-4f8b-4562-8002-76a79a0902f2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080518 Aug 24 10:28:51.692608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 891/1642] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:51 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:52.386274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-36889835-dc0f-4834-9b75-fa6fe4ec27d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:52.388986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-36889835-dc0f-4834-9b75-fa6fe4ec27d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:52.390899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=84 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:52.396143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36889835-dc0f-4834-9b75-fa6fe4ec27d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:52.396631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36889835-dc0f-4834-9b75-fa6fe4ec27d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:52.397073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36889835-dc0f-4834-9b75-fa6fe4ec27d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:52.397760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36889835-dc0f-4834-9b75-fa6fe4ec27d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:52.398215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36889835-dc0f-4834-9b75-fa6fe4ec27d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:52.398640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36889835-dc0f-4834-9b75-fa6fe4ec27d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:52.427934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-36889835-dc0f-4834-9b75-fa6fe4ec27d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045679 Aug 24 10:28:52.428561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 752/1643] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:52 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:52.579442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:52.580087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:52.586380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:52.586866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:52.587296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:52.587972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:52.588444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:52.588971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:52.712838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:52.713564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:52.717308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:52.720381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:52.720869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:52.721317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:52.722018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:52.722470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:52.722896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:52.725979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:52.727478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:52.728666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a03eceeb-cd84-4153-8e05-f1ae2aa1f22c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153102 Aug 24 10:28:52.729213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 892/1644] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:52 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:52.784580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:52.792334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-46341207-8030-48f2-bd81-9c1b6dffa098 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083306 Aug 24 10:28:52.792899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 753/1645] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:52 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:53.440099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6048a8f1-6fff-416d-9c2e-a0d1526245b3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:53.442574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6048a8f1-6fff-416d-9c2e-a0d1526245b3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:53.444460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=80 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:53.450084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6048a8f1-6fff-416d-9c2e-a0d1526245b3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:53.450560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6048a8f1-6fff-416d-9c2e-a0d1526245b3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:53.450993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6048a8f1-6fff-416d-9c2e-a0d1526245b3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:53.451676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6048a8f1-6fff-416d-9c2e-a0d1526245b3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:53.452157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6048a8f1-6fff-416d-9c2e-a0d1526245b3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:53.452579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6048a8f1-6fff-416d-9c2e-a0d1526245b3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:53.483134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6048a8f1-6fff-416d-9c2e-a0d1526245b3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047047 Aug 24 10:28:53.483666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 893/1646] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:53.750865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:53.751476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:53.757941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:53.758419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:53.758852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:53.759534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:53.759983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:53.760428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:53.812640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:53.813371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:53.819792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:53.820401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:53.820908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:53.821681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:53.822490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:53.822986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:53.884556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:53.892873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e847c3c7-2ed0-4621-b77e-33e2e4790913 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084010 Aug 24 10:28:53.893476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 894/1647] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:53 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:53.895635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:53.905114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:53.906739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:53.907905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ee6eef6a-9152-422d-98ee-9a195e8d9b84 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.160974 Aug 24 10:28:53.908513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 754/1648] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:53 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:54.495221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e86b62e7-b7cb-428a-bc4e-33ec8e65e13e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:54.497965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e86b62e7-b7cb-428a-bc4e-33ec8e65e13e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:54.503897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86b62e7-b7cb-428a-bc4e-33ec8e65e13e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:54.504417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86b62e7-b7cb-428a-bc4e-33ec8e65e13e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:54.504858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86b62e7-b7cb-428a-bc4e-33ec8e65e13e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:54.505552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86b62e7-b7cb-428a-bc4e-33ec8e65e13e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:54.506007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86b62e7-b7cb-428a-bc4e-33ec8e65e13e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:54.506436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e86b62e7-b7cb-428a-bc4e-33ec8e65e13e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:54.536880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e86b62e7-b7cb-428a-bc4e-33ec8e65e13e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045630 Aug 24 10:28:54.537481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 895/1649] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:54 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:54.913658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:54.914252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:54.921269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:54.921741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:54.922169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:54.922846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:54.923292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:54.923707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:54.930560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:54.931175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:54.937860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:54.938350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:54.938791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:54.939484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:54.939944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:54.940404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:54.985948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:54.993584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f3715cb2-be2d-4c69-b389-1834751320b3 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083775 Aug 24 10:28:54.994139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 755/1650] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:54 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:28:55.069395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:55.078991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:55.080499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:55.081651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c6b6564c-3ba3-4d40-9f09-43ece0e9b213 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155005 Aug 24 10:28:55.082196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 896/1651] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:54 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:28:55.549986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b5bf9797-70f6-469f-a5e9-f05753ebbb78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:55.552206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b5bf9797-70f6-469f-a5e9-f05753ebbb78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:55.558110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5bf9797-70f6-469f-a5e9-f05753ebbb78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:55.558627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5bf9797-70f6-469f-a5e9-f05753ebbb78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:55.559058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5bf9797-70f6-469f-a5e9-f05753ebbb78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:55.559734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5bf9797-70f6-469f-a5e9-f05753ebbb78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:55.560214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5bf9797-70f6-469f-a5e9-f05753ebbb78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:55.560634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5bf9797-70f6-469f-a5e9-f05753ebbb78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:55.589139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b5bf9797-70f6-469f-a5e9-f05753ebbb78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043178 Aug 24 10:28:55.589670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 756/1652] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:55 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:28:56.014708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:56.015307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:56.021742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:56.022217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:56.022649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:56.023329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:56.023778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:56.024226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:56.088299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:56.096556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b0697dfb-37c8-46b0-82fa-2cc8f5601aeb tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086162 Aug 24 10:28:56.097107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 897/1653] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:56 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:56.104167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:56.104780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:56.111298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:56.112400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:56.112872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:56.113587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:56.114051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:56.114476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:56.249679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:56.258253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:56.259744 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:56.260911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-17eb0bb3-12a3-4977-8a3f-f322a4e2ae41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.160757 Aug 24 10:28:56.261461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 757/1654] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:56.601352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cd1d9942-f4e0-463b-8d55-0b72c0eff1d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:56.603780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cd1d9942-f4e0-463b-8d55-0b72c0eff1d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:56.610398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd1d9942-f4e0-463b-8d55-0b72c0eff1d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:56.610815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd1d9942-f4e0-463b-8d55-0b72c0eff1d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:56.611249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd1d9942-f4e0-463b-8d55-0b72c0eff1d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:56.611939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd1d9942-f4e0-463b-8d55-0b72c0eff1d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:56.612420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd1d9942-f4e0-463b-8d55-0b72c0eff1d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:56.612844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd1d9942-f4e0-463b-8d55-0b72c0eff1d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:56.642093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cd1d9942-f4e0-463b-8d55-0b72c0eff1d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044777 Aug 24 10:28:56.642673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 898/1655] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:56 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:57.119104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:57.119905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:57.127019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:57.127495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:57.127922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:57.128634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:57.129079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:57.129495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:57.191230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:57.199928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5bb77bd2-079a-431f-a49c-6cb7d3202a64 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085329 Aug 24 10:28:57.200522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 758/1656] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:57 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:57.284968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:57.285585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:57.291978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:57.292510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:57.292947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:57.293636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:57.294090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:57.294517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:57.419563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:57.428242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:57.429794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:57.430948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bfea7856-638a-46b4-bee5-ee8e4171563f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.149994 Aug 24 10:28:57.431496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 899/1657] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:57 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:57.655135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d21a1595-0e98-4aa6-b5bf-71de5bad1f61 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:57.657284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d21a1595-0e98-4aa6-b5bf-71de5bad1f61 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:57.663242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d21a1595-0e98-4aa6-b5bf-71de5bad1f61 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:57.663723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d21a1595-0e98-4aa6-b5bf-71de5bad1f61 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:57.664189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d21a1595-0e98-4aa6-b5bf-71de5bad1f61 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:57.664887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d21a1595-0e98-4aa6-b5bf-71de5bad1f61 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:57.665367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d21a1595-0e98-4aa6-b5bf-71de5bad1f61 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:57.665791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d21a1595-0e98-4aa6-b5bf-71de5bad1f61 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:57.693853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d21a1595-0e98-4aa6-b5bf-71de5bad1f61 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043396 Aug 24 10:28:57.694405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 759/1658] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:57 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:58.221142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:58.221753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:58.227691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:58.228225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:58.228658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:58.229346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:58.229792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:58.230210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:58.292976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:58.300715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d4e072a8-3c6c-459a-8edd-9bb4c199fc7d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083606 Aug 24 10:28:58.301273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 900/1659] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:58 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:58.454964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:58.455573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:58.461952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:58.462426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:58.462857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:58.463533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:58.463979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:58.464421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:58.592262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:58.600782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:58.602268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:58.603405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-34a41637-de9f-4d79-852b-8599d76cdeb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152493 Aug 24 10:28:58.603963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 760/1660] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:58 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:58.706688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0ed49597-6cf7-4abc-9b9d-c0a646e711c8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:58.708826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0ed49597-6cf7-4abc-9b9d-c0a646e711c8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:58.714678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ed49597-6cf7-4abc-9b9d-c0a646e711c8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:58.715160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ed49597-6cf7-4abc-9b9d-c0a646e711c8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:58.715595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ed49597-6cf7-4abc-9b9d-c0a646e711c8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:58.716306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ed49597-6cf7-4abc-9b9d-c0a646e711c8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:58.716762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ed49597-6cf7-4abc-9b9d-c0a646e711c8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:58.717185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ed49597-6cf7-4abc-9b9d-c0a646e711c8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:58.745254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0ed49597-6cf7-4abc-9b9d-c0a646e711c8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042471 Aug 24 10:28:58.746279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 901/1661] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:58 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:28:59.321218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:59.322085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:59.328036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:59.328541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:59.328974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:59.329656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:59.330110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:59.330542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:59.348307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:28:59.392634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:59.400169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5316a137-343a-48c3-92c7-a7321f4adeb4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082866 Aug 24 10:28:59.400727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 761/1662] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:28:59 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:28:59.626066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:59.626577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:59.632934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:59.633413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:59.633848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:59.634539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:59.634994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:59.635418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:59.759028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-afa70870-f895-40c6-8251-68a5363a2842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:28:59.761925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-afa70870-f895-40c6-8251-68a5363a2842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:28:59.764000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:59.769494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-afa70870-f895-40c6-8251-68a5363a2842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:59.769985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-afa70870-f895-40c6-8251-68a5363a2842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:59.770423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-afa70870-f895-40c6-8251-68a5363a2842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:59.771165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-afa70870-f895-40c6-8251-68a5363a2842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:28:59.771616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-afa70870-f895-40c6-8251-68a5363a2842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:28:59.772038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-afa70870-f895-40c6-8251-68a5363a2842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:28:59.774369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:59.775926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:28:59.777901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a9cec84d-7c02-4569-80d3-b9ac80c515e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155072 Aug 24 10:28:59.778313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 902/1663] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:28:59 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:28:59.804204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-afa70870-f895-40c6-8251-68a5363a2842 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049757 Aug 24 10:28:59.804756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 762/1664] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:28:59 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:00.421947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:00.422565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:00.428560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:00.429051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:00.429493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:00.430186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:00.430642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:00.431074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:00.491285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:00.499517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f7431f88-bc97-44ba-810d-8443046b9b19 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081957 Aug 24 10:29:00.500086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 903/1665] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:00 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:00.799843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:00.800474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:00.806846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:00.807320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:00.807750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:00.808453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:00.808904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:00.809319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:00.815513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7dfb751a-e1ec-44d9-8000-7f2185fc795c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:00.817700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7dfb751a-e1ec-44d9-8000-7f2185fc795c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:00.824444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dfb751a-e1ec-44d9-8000-7f2185fc795c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:00.824935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dfb751a-e1ec-44d9-8000-7f2185fc795c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:00.825634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dfb751a-e1ec-44d9-8000-7f2185fc795c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:00.826134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dfb751a-e1ec-44d9-8000-7f2185fc795c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:00.826589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dfb751a-e1ec-44d9-8000-7f2185fc795c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:00.827020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dfb751a-e1ec-44d9-8000-7f2185fc795c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:00.857810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7dfb751a-e1ec-44d9-8000-7f2185fc795c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046009 Aug 24 10:29:00.858580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 904/1666] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:00 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:00.942093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:00.950973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:00.952486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:00.953633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-97e4868d-738e-43f5-a972-457d1575887a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157680 Aug 24 10:29:00.954182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 763/1667] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:00 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:01.520180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:01.520799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:01.526843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:01.527332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:01.527772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:01.528545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:01.528956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:01.529387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:01.533390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:01.589846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:01.597787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-28aa7725-d2be-4625-b98e-6e0304ce9695 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081573 Aug 24 10:29:01.598344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 905/1668] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:01 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:01.880303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4f5b428c-3aff-453a-bfec-a4c05ba3d7da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:01.883150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4f5b428c-3aff-453a-bfec-a4c05ba3d7da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:01.889956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f5b428c-3aff-453a-bfec-a4c05ba3d7da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:01.890445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f5b428c-3aff-453a-bfec-a4c05ba3d7da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:01.890874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f5b428c-3aff-453a-bfec-a4c05ba3d7da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:01.891566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f5b428c-3aff-453a-bfec-a4c05ba3d7da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:01.892023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f5b428c-3aff-453a-bfec-a4c05ba3d7da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:01.892575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f5b428c-3aff-453a-bfec-a4c05ba3d7da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:01.922613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4f5b428c-3aff-453a-bfec-a4c05ba3d7da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046652 Aug 24 10:29:01.923347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 764/1669] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:01 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:01.976340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:01.976953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:01.983766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:01.984292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:01.984854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:01.985545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:01.985997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:01.986421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:02.123062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:02.131895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:02.133364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:02.134505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8c350ea2-d678-4af1-9810-12c307ab67ba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.162227 Aug 24 10:29:02.135057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 906/1670] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:29:02.618464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:02.619095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:02.626040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:02.626520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:02.626950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:02.627629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:02.628102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:02.628519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:02.688626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:02.696353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-efc8a0d2-8093-4578-8eeb-2744e2731497 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081763 Aug 24 10:29:02.697183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 765/1671] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:02 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:02.934413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4e4304a1-f2fa-406f-951b-0092df1095e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:02.936597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4e4304a1-f2fa-406f-951b-0092df1095e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:02.942500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4304a1-f2fa-406f-951b-0092df1095e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:02.942977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4304a1-f2fa-406f-951b-0092df1095e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:02.943410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4304a1-f2fa-406f-951b-0092df1095e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:02.944110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4304a1-f2fa-406f-951b-0092df1095e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:02.944595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4304a1-f2fa-406f-951b-0092df1095e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:02.945021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e4304a1-f2fa-406f-951b-0092df1095e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:02.975121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4e4304a1-f2fa-406f-951b-0092df1095e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044589 Aug 24 10:29:02.975691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 907/1672] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:02 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:03.157476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:03.158213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:03.164661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:03.165130 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:03.165555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:03.166226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:03.166671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:03.167084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:03.297397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:03.305975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:03.307464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:03.309169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9f369c37-7a42-4a09-9832-3dd5e9902ea1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155574 Aug 24 10:29:03.309727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 766/1673] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:03.782228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:03.782858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:03.789035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:03.789523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:03.789957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:03.790645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:03.791098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:03.791521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:03.852340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:03.860708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-979dadc7-1fdb-4bac-b6db-e4dac7da1ba0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.146901 Aug 24 10:29:03.861268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 908/1674] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:03 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:03.987469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0cbd54a8-a0b7-4ce3-ae2f-71596f57c202 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:03.989651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0cbd54a8-a0b7-4ce3-ae2f-71596f57c202 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:03.995741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cbd54a8-a0b7-4ce3-ae2f-71596f57c202 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:03.996298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cbd54a8-a0b7-4ce3-ae2f-71596f57c202 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:03.996760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cbd54a8-a0b7-4ce3-ae2f-71596f57c202 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:03.997624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cbd54a8-a0b7-4ce3-ae2f-71596f57c202 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:03.998081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cbd54a8-a0b7-4ce3-ae2f-71596f57c202 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:03.998550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cbd54a8-a0b7-4ce3-ae2f-71596f57c202 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:04.027955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0cbd54a8-a0b7-4ce3-ae2f-71596f57c202 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044604 Aug 24 10:29:04.028536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 767/1675] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:03 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:04.332950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:04.333520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:04.340153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:04.340656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:04.341078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:04.341766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:04.342221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:04.342648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:04.470434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:04.479114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:04.480734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:04.481889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-543ddb4c-0983-4dac-831e-d3a202ec2392 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152989 Aug 24 10:29:04.482469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 909/1676] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:04.881385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:04.882032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:04.888422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:04.888903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:04.889335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:04.890021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:04.890471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:04.890893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:04.952633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:04.960381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-724d1697-1cff-4873-8ea5-4cc061b10a36 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082806 Aug 24 10:29:04.960956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 768/1677] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:05.040343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b70dd6b2-64ab-4201-a55a-c68f1fd52341 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:05.043046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b70dd6b2-64ab-4201-a55a-c68f1fd52341 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:05.049492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b70dd6b2-64ab-4201-a55a-c68f1fd52341 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:05.049967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b70dd6b2-64ab-4201-a55a-c68f1fd52341 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:05.050404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b70dd6b2-64ab-4201-a55a-c68f1fd52341 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:05.051092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b70dd6b2-64ab-4201-a55a-c68f1fd52341 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:05.051551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b70dd6b2-64ab-4201-a55a-c68f1fd52341 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:05.051979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b70dd6b2-64ab-4201-a55a-c68f1fd52341 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:05.081604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b70dd6b2-64ab-4201-a55a-c68f1fd52341 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045454 Aug 24 10:29:05.082153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 910/1678] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:05 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:05.505616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:05.506252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:05.512345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:05.512845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:05.513263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:05.513950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:05.514400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:05.514821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:05.645656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:05.655013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:05.656546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:05.657751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bcb68bb7-7008-46e3-a8e2-95b9905bc972 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156075 Aug 24 10:29:05.658316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 769/1679] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:05.980755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:05.981371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:05.987553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:05.988038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:05.988508 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:05.989199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:05.989650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:05.990075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:06.050795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:06.058969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bb9a8787-fef4-4dcd-97c8-5e40e3e41590 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082039 Aug 24 10:29:06.059521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 911/1680] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:06.094369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-205e06ef-01d0-48a6-8b86-9e5651d3f6d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:06.096548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-205e06ef-01d0-48a6-8b86-9e5651d3f6d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:06.102931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-205e06ef-01d0-48a6-8b86-9e5651d3f6d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:06.103416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-205e06ef-01d0-48a6-8b86-9e5651d3f6d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:06.103854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-205e06ef-01d0-48a6-8b86-9e5651d3f6d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:06.104573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-205e06ef-01d0-48a6-8b86-9e5651d3f6d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:06.105032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-205e06ef-01d0-48a6-8b86-9e5651d3f6d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:06.105454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-205e06ef-01d0-48a6-8b86-9e5651d3f6d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:06.134374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-205e06ef-01d0-48a6-8b86-9e5651d3f6d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043894 Aug 24 10:29:06.134933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 770/1681] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:06 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:06.680455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:06.680991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:06.687490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:06.687969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:06.688441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:06.689125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:06.689573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:06.689994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:07.027796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:07.036593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:07.038091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:07.039234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b3702e01-7d1e-45a8-900e-a420262a6572 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.362816 Aug 24 10:29:07.039776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 912/1682] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:07.080195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:07.080817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:07.086810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:07.087292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:07.087730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:07.088461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:07.088901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:07.089826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:07.146921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-632ec6f4-fb3a-471c-ab72-8206959eb61e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:07.149297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-632ec6f4-fb3a-471c-ab72-8206959eb61e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:07.153367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:07.155880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632ec6f4-fb3a-471c-ab72-8206959eb61e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:07.156402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632ec6f4-fb3a-471c-ab72-8206959eb61e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:07.156845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632ec6f4-fb3a-471c-ab72-8206959eb61e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:07.157544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632ec6f4-fb3a-471c-ab72-8206959eb61e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:07.158021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632ec6f4-fb3a-471c-ab72-8206959eb61e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:07.158452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-632ec6f4-fb3a-471c-ab72-8206959eb61e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:07.161133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-24c901ff-4a0d-49f1-b143-332131768270 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084921 Aug 24 10:29:07.161929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 771/1683] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:07 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:07.187823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-632ec6f4-fb3a-471c-ab72-8206959eb61e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044858 Aug 24 10:29:07.188458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 913/1684] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:07 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:08.063178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:08.063709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:08.069705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:08.070189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:08.070626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:08.071313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:08.072020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:08.072478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:08.181701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:08.182327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:08.189364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:08.189864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:08.190308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:08.191018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:08.191473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:08.191905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:08.201912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:08.210733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:08.212266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:08.213437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ce47bb70-c5c5-4d36-9bc4-392c43fa2f81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154325 Aug 24 10:29:08.214000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 772/1685] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:08 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:08.218512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-77f424b5-5076-4ee3-b2f2-69800f7b6007 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:08.220712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-77f424b5-5076-4ee3-b2f2-69800f7b6007 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:08.226580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77f424b5-5076-4ee3-b2f2-69800f7b6007 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:08.227069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77f424b5-5076-4ee3-b2f2-69800f7b6007 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:08.227508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77f424b5-5076-4ee3-b2f2-69800f7b6007 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:08.228235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77f424b5-5076-4ee3-b2f2-69800f7b6007 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:08.228909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77f424b5-5076-4ee3-b2f2-69800f7b6007 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:08.229340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77f424b5-5076-4ee3-b2f2-69800f7b6007 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:08.254735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:08.262890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-77f424b5-5076-4ee3-b2f2-69800f7b6007 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047855 Aug 24 10:29:08.263786 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 773/1686] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:08 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:08.271300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ddfc11fb-ced0-4686-a9bb-af65a8d89c35 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.093434 Aug 24 10:29:08.271898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 914/1687] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:08 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:09.236797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:09.237400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:09.243716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:09.244228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:09.244658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:09.245820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:09.246263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:09.246679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:09.275258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-101d4df6-0947-4d93-a84f-623f59ca18d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:09.277454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-101d4df6-0947-4d93-a84f-623f59ca18d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:09.284032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-101d4df6-0947-4d93-a84f-623f59ca18d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:09.284541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-101d4df6-0947-4d93-a84f-623f59ca18d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:09.284985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-101d4df6-0947-4d93-a84f-623f59ca18d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:09.287732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-101d4df6-0947-4d93-a84f-623f59ca18d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:09.287732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-101d4df6-0947-4d93-a84f-623f59ca18d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:09.287732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-101d4df6-0947-4d93-a84f-623f59ca18d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:09.316687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-101d4df6-0947-4d93-a84f-623f59ca18d4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045069 Aug 24 10:29:09.317251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 915/1688] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:09 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:09.321880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:09.322488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:09.328889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:09.329370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:09.329805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:09.330487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:09.330935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:09.331359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:09.378630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:09.387283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:09.388838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:09.390000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-24981bd1-c221-468b-9310-963ae3a902bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157152 Aug 24 10:29:09.390561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 774/1689] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:09 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:09.392316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:09.400022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5dcf09a3-615e-4ee1-930e-3d4898dce2b0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081823 Aug 24 10:29:09.400599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 916/1690] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:09 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:10.329630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f879628f-4bfb-46b2-ac3f-c86550f48e4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:10.331795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f879628f-4bfb-46b2-ac3f-c86550f48e4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:10.338303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f879628f-4bfb-46b2-ac3f-c86550f48e4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:10.338730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f879628f-4bfb-46b2-ac3f-c86550f48e4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:10.339165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f879628f-4bfb-46b2-ac3f-c86550f48e4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:10.339856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f879628f-4bfb-46b2-ac3f-c86550f48e4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:10.340337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f879628f-4bfb-46b2-ac3f-c86550f48e4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:10.340769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f879628f-4bfb-46b2-ac3f-c86550f48e4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:10.370518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f879628f-4bfb-46b2-ac3f-c86550f48e4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045528 Aug 24 10:29:10.371090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 775/1691] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:10 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:10.412671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:10.413280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:10.419490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:10.419669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:10.420184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:10.420293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:10.420619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:10.421312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:10.421761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:10.422190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:10.426032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:10.426520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:10.426964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:10.427675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:10.428137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:10.428563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:10.493360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:10.501642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dfdf989c-e7bc-4917-b5a4-6313e65dde92 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085908 Aug 24 10:29:10.502425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 776/1692] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:10 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:10.556963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:10.568368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:10.569956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:10.571184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fc163239-863f-4d27-9cc9-fdf7ab4be1f4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.162308 Aug 24 10:29:10.572009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 917/1693] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:10 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:11.383039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-955499e8-a478-4891-b680-aa7538282837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:11.385206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-955499e8-a478-4891-b680-aa7538282837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:11.391682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955499e8-a478-4891-b680-aa7538282837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:11.392201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955499e8-a478-4891-b680-aa7538282837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:11.392635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955499e8-a478-4891-b680-aa7538282837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:11.393326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955499e8-a478-4891-b680-aa7538282837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:11.393773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955499e8-a478-4891-b680-aa7538282837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:11.394191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-955499e8-a478-4891-b680-aa7538282837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:11.424885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-955499e8-a478-4891-b680-aa7538282837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045906 Aug 24 10:29:11.425455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 777/1694] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:11 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:11.524949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:11.525701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:11.535954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:11.536398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:11.536940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:11.537732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:11.538394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:11.539139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:11.597768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:11.598380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:11.605004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:11.605594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:11.606029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:11.606727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:11.607278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:11.607695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:11.610774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:11.614033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:11.620483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-db2d967e-49c0-4c1b-abeb-2cdcae723705 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.100645 Aug 24 10:29:11.621076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 918/1695] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:11 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:11.748307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:11.757019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:11.758514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:11.759655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bf55e478-6d8e-4dd8-a124-5e4d5fcde95e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.168189 Aug 24 10:29:11.760233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 778/1696] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:11 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:29:12.437888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8c78137f-c15a-46ce-be4a-3c961e30dd14 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:12.440026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8c78137f-c15a-46ce-be4a-3c961e30dd14 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:12.445971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c78137f-c15a-46ce-be4a-3c961e30dd14 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:12.446723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c78137f-c15a-46ce-be4a-3c961e30dd14 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:12.447164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c78137f-c15a-46ce-be4a-3c961e30dd14 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:12.447854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c78137f-c15a-46ce-be4a-3c961e30dd14 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:12.448349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c78137f-c15a-46ce-be4a-3c961e30dd14 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:12.448781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8c78137f-c15a-46ce-be4a-3c961e30dd14 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:12.476956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8c78137f-c15a-46ce-be4a-3c961e30dd14 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043138 Aug 24 10:29:12.477500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 919/1697] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:12 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:12.641793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:12.642395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:12.649239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:12.649719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:12.650151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:12.650874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:12.651325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:12.651742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:12.713771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:12.721474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c74446fe-80d4-4a7e-818a-b6609620c736 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083561 Aug 24 10:29:12.722040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 779/1698] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:12 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:12.783246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:12.783866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:12.790477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:12.790967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:12.791410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:12.792813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:12.793276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:12.793708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:12.919259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:12.927891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:12.929429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:12.930587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-52a85812-a047-41bf-8ad6-201e8af4dd5b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151300 Aug 24 10:29:12.931144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 920/1699] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:12 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:13.489722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dc7e2e45-ef4d-43ab-bd19-9def369b9793 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:13.491871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dc7e2e45-ef4d-43ab-bd19-9def369b9793 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:13.498521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc7e2e45-ef4d-43ab-bd19-9def369b9793 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:13.499010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc7e2e45-ef4d-43ab-bd19-9def369b9793 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:13.499445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc7e2e45-ef4d-43ab-bd19-9def369b9793 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:13.500165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc7e2e45-ef4d-43ab-bd19-9def369b9793 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:13.500624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc7e2e45-ef4d-43ab-bd19-9def369b9793 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:13.501053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc7e2e45-ef4d-43ab-bd19-9def369b9793 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:13.531370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dc7e2e45-ef4d-43ab-bd19-9def369b9793 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045684 Aug 24 10:29:13.531924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 780/1700] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:13 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:13.742504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:13.743123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:13.749601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:13.750082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:13.750518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:13.751206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:13.751675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:13.752107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:13.815915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:13.819409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:13.825319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-48b68288-119b-4b7c-a6f2-a18798bc3a93 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086686 Aug 24 10:29:13.825874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 921/1701] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:13 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:13.953243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:13.953879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:13.960330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:13.960837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:13.961271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:13.961951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:13.962402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:13.962821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:14.095283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:14.103946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:14.105965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:14.107114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c7095491-0ff1-4898-9b2c-fa9a6292b36e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157797 Aug 24 10:29:14.107671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 781/1702] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:14.543636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3d3d3d52-e0ab-4170-8701-630ba78abab3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:14.545829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3d3d3d52-e0ab-4170-8701-630ba78abab3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:14.552145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d3d3d52-e0ab-4170-8701-630ba78abab3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:14.552640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d3d3d52-e0ab-4170-8701-630ba78abab3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:14.553083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d3d3d52-e0ab-4170-8701-630ba78abab3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:14.553780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d3d3d52-e0ab-4170-8701-630ba78abab3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:14.554239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d3d3d52-e0ab-4170-8701-630ba78abab3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:14.554674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d3d3d52-e0ab-4170-8701-630ba78abab3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:14.583847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3d3d3d52-e0ab-4170-8701-630ba78abab3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044855 Aug 24 10:29:14.584424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 922/1703] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:14 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:14.845567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:14.846091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:14.851915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:14.852425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:14.852862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:14.853540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:14.853985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:14.854400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:14.914899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:14.922685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-57f99d51-0666-417e-925e-3866ea509fe0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081059 Aug 24 10:29:14.923246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 782/1704] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:14 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:15.130529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:15.131096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:15.138177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:15.138663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:15.139105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:15.139800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:15.140291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:15.140722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:15.266618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:15.275372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:15.276910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:15.278113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-415e0dd4-8d2a-4385-8033-5909233a8277 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151605 Aug 24 10:29:15.278762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 923/1705] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:15.596229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-38f0795a-addc-4279-aef0-6ada55b2d668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:15.598437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-38f0795a-addc-4279-aef0-6ada55b2d668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:15.605424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38f0795a-addc-4279-aef0-6ada55b2d668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:15.605906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38f0795a-addc-4279-aef0-6ada55b2d668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:15.606342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38f0795a-addc-4279-aef0-6ada55b2d668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:15.607034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38f0795a-addc-4279-aef0-6ada55b2d668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:15.607489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38f0795a-addc-4279-aef0-6ada55b2d668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:15.607917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38f0795a-addc-4279-aef0-6ada55b2d668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:15.637444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-38f0795a-addc-4279-aef0-6ada55b2d668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045166 Aug 24 10:29:15.637972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 783/1706] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:15 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:15.943200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:15.943848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:15.950111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:15.950588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:15.951021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:15.954121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:15.954609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:15.955045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:16.016384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:16.024039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-59d5bda8-3960-4145-a6a6-a55b4473ae83 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084708 Aug 24 10:29:16.024611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 924/1707] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:15 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:16.301544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:16.302096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:16.308057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:16.308548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:16.308988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:16.309674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:16.310131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:16.310554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:16.440522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:16.450043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:16.451554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:16.452748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4ca0dd68-1cb4-4526-9844-5c2b9a30dd92 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155269 Aug 24 10:29:16.453300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 784/1708] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:16 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:16.649827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-942c9788-78f5-4502-b0c2-cb29203ec095 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:16.652063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-942c9788-78f5-4502-b0c2-cb29203ec095 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:16.658499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-942c9788-78f5-4502-b0c2-cb29203ec095 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:16.658987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-942c9788-78f5-4502-b0c2-cb29203ec095 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:16.659432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-942c9788-78f5-4502-b0c2-cb29203ec095 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:16.660164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-942c9788-78f5-4502-b0c2-cb29203ec095 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:16.660627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-942c9788-78f5-4502-b0c2-cb29203ec095 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:16.661057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-942c9788-78f5-4502-b0c2-cb29203ec095 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:16.691330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-942c9788-78f5-4502-b0c2-cb29203ec095 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045521 Aug 24 10:29:16.691911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 925/1709] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:16 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:17.045529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:17.046218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:17.052651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:17.053135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:17.053570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:17.054258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:17.054711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:17.055133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:17.122947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:17.132665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-eb96c7e8-96b9-41c6-9dc5-5b536fe22914 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.091546 Aug 24 10:29:17.133274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 785/1710] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:17 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:17.475293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:17.475909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:17.482539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:17.483027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:17.483470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:17.484197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:17.484655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:17.485085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:17.613289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:17.622291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:17.623781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:17.624975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-22445e8b-4999-4f24-8d2c-5af7910b40b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153626 Aug 24 10:29:17.625524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 926/1711] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:17.704695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8bc05871-17f4-491f-86f4-69dc74819bd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:17.706818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8bc05871-17f4-491f-86f4-69dc74819bd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:17.712771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bc05871-17f4-491f-86f4-69dc74819bd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:17.713252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bc05871-17f4-491f-86f4-69dc74819bd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:17.714260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bc05871-17f4-491f-86f4-69dc74819bd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:17.714952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bc05871-17f4-491f-86f4-69dc74819bd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:17.715404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bc05871-17f4-491f-86f4-69dc74819bd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:17.715825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bc05871-17f4-491f-86f4-69dc74819bd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:17.744196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8bc05871-17f4-491f-86f4-69dc74819bd8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043372 Aug 24 10:29:17.744677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 786/1712] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:17 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:18.153579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:18.154127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:18.162088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:18.162577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:18.163021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:18.163719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:18.164204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:18.164636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:18.226426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:18.234186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a7f05109-01b4-4dcb-9db0-981171c7896f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084546 Aug 24 10:29:18.234740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 927/1713] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:18 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:18.648621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:18.649261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:18.655674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:18.656186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:18.656629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:18.657329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:18.657778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:18.658204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:18.756613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-07450b73-0f85-4995-a89f-b9420aef00fd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:18.758857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-07450b73-0f85-4995-a89f-b9420aef00fd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:18.764711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07450b73-0f85-4995-a89f-b9420aef00fd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:18.765194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07450b73-0f85-4995-a89f-b9420aef00fd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:18.766815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07450b73-0f85-4995-a89f-b9420aef00fd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:18.766815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07450b73-0f85-4995-a89f-b9420aef00fd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:18.766815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07450b73-0f85-4995-a89f-b9420aef00fd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:18.767243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07450b73-0f85-4995-a89f-b9420aef00fd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:18.787987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:18.797620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:18.799127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:18.800101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-07450b73-0f85-4995-a89f-b9420aef00fd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047935 Aug 24 10:29:18.800441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b821fd53-7cff-499f-b872-1368c9cb305f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155706 Aug 24 10:29:18.800668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 928/1714] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:18 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:18.803984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 787/1715] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:18 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:19.255081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:19.255708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:19.262660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:19.263136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:19.263567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:19.264290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:19.264741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:19.265165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:19.325814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:19.333576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-86bd8843-e7e1-426f-b9e3-0b9d2be0eafe tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082374 Aug 24 10:29:19.334138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 929/1716] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:19 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:19.813216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2fe2fcdd-9547-4094-9b1a-d3fcf3fd798c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:19.815147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2fe2fcdd-9547-4094-9b1a-d3fcf3fd798c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:19.821574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fe2fcdd-9547-4094-9b1a-d3fcf3fd798c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:19.822053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fe2fcdd-9547-4094-9b1a-d3fcf3fd798c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:19.822486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fe2fcdd-9547-4094-9b1a-d3fcf3fd798c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:19.823652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:19.823882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fe2fcdd-9547-4094-9b1a-d3fcf3fd798c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:19.824209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:19.824351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fe2fcdd-9547-4094-9b1a-d3fcf3fd798c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:19.824708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fe2fcdd-9547-4094-9b1a-d3fcf3fd798c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:19.830072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:19.830581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:19.831021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:19.831717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:19.832217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:19.832641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:19.853694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2fe2fcdd-9547-4094-9b1a-d3fcf3fd798c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044700 Aug 24 10:29:19.854769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 788/1717] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:19 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:19.961442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:19.970240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:19.971774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:19.972950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-121b45b3-e609-41f3-a705-234177c43426 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154208 Aug 24 10:29:19.973506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 930/1718] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:29:20.355268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:20.355873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:20.361821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:20.362300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:20.362730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:20.363410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:20.363860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:20.364318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:20.425677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:20.433323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c6e26d03-0aa9-441e-912d-bd3530457813 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082569 Aug 24 10:29:20.433880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 789/1719] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:20 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:20.866512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-58519a86-c5e3-469e-a14d-5f881d2dd929 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:20.868669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-58519a86-c5e3-469e-a14d-5f881d2dd929 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:20.874948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58519a86-c5e3-469e-a14d-5f881d2dd929 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:20.875497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58519a86-c5e3-469e-a14d-5f881d2dd929 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:20.875915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58519a86-c5e3-469e-a14d-5f881d2dd929 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:20.876568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58519a86-c5e3-469e-a14d-5f881d2dd929 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:20.877017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58519a86-c5e3-469e-a14d-5f881d2dd929 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:20.877438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58519a86-c5e3-469e-a14d-5f881d2dd929 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:20.906674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-58519a86-c5e3-469e-a14d-5f881d2dd929 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044131 Aug 24 10:29:20.907221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 931/1720] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:20 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:20.996679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:20.997323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:21.003749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:21.004281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:21.004718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:21.005414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:21.005865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:21.006291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:21.138282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:21.146802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:21.148328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:21.149486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-82346cfe-5da3-4611-8f9b-e183014ff1be tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157341 Aug 24 10:29:21.150047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 790/1721] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:20 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:21.454243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:21.454897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:21.460978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:21.461461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:21.461902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:21.462593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:21.463055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:21.463483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:21.522872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:21.530524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b78012bc-591c-4141-8375-c5f316289df1 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.080185 Aug 24 10:29:21.531084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 932/1722] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:21 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:21.919592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-02f5a018-40c0-450d-bde9-597d4872a318 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:21.921728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-02f5a018-40c0-450d-bde9-597d4872a318 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:21.927656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f5a018-40c0-450d-bde9-597d4872a318 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:21.928164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f5a018-40c0-450d-bde9-597d4872a318 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:21.928601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f5a018-40c0-450d-bde9-597d4872a318 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:21.929278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f5a018-40c0-450d-bde9-597d4872a318 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:21.929730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f5a018-40c0-450d-bde9-597d4872a318 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:21.930147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f5a018-40c0-450d-bde9-597d4872a318 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:21.958668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-02f5a018-40c0-450d-bde9-597d4872a318 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043666 Aug 24 10:29:21.959219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 791/1723] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:21 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:22.172382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:22.172940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:22.179282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:22.179769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:22.180248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:22.180945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:22.181412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:22.181845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:22.307452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:22.316211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:22.317724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:22.318883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-deb76051-08d3-46ba-b727-8a4b310a9cea tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.150533 Aug 24 10:29:22.319436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 933/1724] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:22 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:22.551738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:22.552420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:22.554671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=80 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:22.560358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:22.560844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:22.561285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:22.561969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:22.562425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:22.562851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:22.623836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:22.631528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fb92d1bb-00bc-4ec4-be6d-c9c5f104d6ad tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083625 Aug 24 10:29:22.632106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 792/1725] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:22 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:22.970899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5ddbac63-d6db-4cc3-8d9b-9262936569cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:22.973094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5ddbac63-d6db-4cc3-8d9b-9262936569cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:22.979111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ddbac63-d6db-4cc3-8d9b-9262936569cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:22.979599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ddbac63-d6db-4cc3-8d9b-9262936569cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:22.980062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ddbac63-d6db-4cc3-8d9b-9262936569cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:22.980763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ddbac63-d6db-4cc3-8d9b-9262936569cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:22.981224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ddbac63-d6db-4cc3-8d9b-9262936569cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:22.981660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ddbac63-d6db-4cc3-8d9b-9262936569cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:23.011322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5ddbac63-d6db-4cc3-8d9b-9262936569cf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044386 Aug 24 10:29:23.011878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 934/1726] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:22 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:23.342709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:23.343334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:23.349520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:23.350008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:23.350448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:23.351141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:23.351594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:23.352018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:23.477472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:23.486135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:23.487646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:23.488924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-100443a8-50ad-4172-bf0c-11def034791a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.150169 Aug 24 10:29:23.489481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 793/1727] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:23 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:23.652911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:23.653483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:23.655784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=84 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:23.661063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:23.661546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:23.661983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:23.662669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:23.663123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:23.663548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:23.727503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:23.735670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-88ab0b98-cf37-4f5a-9847-ea71f258eca8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086712 Aug 24 10:29:23.736260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 935/1728] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:23 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:24.023250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-80058621-a020-4de3-b5f9-496e902a1989 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:24.025985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-80058621-a020-4de3-b5f9-496e902a1989 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:24.032494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80058621-a020-4de3-b5f9-496e902a1989 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:24.032977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80058621-a020-4de3-b5f9-496e902a1989 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:24.033417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80058621-a020-4de3-b5f9-496e902a1989 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:24.034114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80058621-a020-4de3-b5f9-496e902a1989 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:24.034573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80058621-a020-4de3-b5f9-496e902a1989 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:24.034999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80058621-a020-4de3-b5f9-496e902a1989 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:24.059965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:24.064969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-80058621-a020-4de3-b5f9-496e902a1989 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045706 Aug 24 10:29:24.065546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 794/1729] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:24 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:24.512511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:24.513157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:24.519614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:24.520135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:24.520580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:24.521284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:24.521789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:24.522161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:24.654712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:24.663879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:24.665429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:24.666634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-85e2f4e5-a769-4424-b83b-964c5cecc111 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158800 Aug 24 10:29:24.667259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 936/1730] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:24 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:24.756829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:24.757438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:24.764020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:24.764522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:24.764954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:24.765635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:24.766082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:24.766505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:24.831905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:24.839641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-91cebc16-d415-4dc9-9eff-8ae0fa4383c4 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087117 Aug 24 10:29:24.840224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 795/1731] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:24 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:25.078038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-56438049-6356-4889-b74b-514c83726036 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:25.080351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-56438049-6356-4889-b74b-514c83726036 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:25.086219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-56438049-6356-4889-b74b-514c83726036 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:25.086700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-56438049-6356-4889-b74b-514c83726036 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:25.087130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-56438049-6356-4889-b74b-514c83726036 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:25.087839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-56438049-6356-4889-b74b-514c83726036 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:25.088359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-56438049-6356-4889-b74b-514c83726036 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:25.088783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-56438049-6356-4889-b74b-514c83726036 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:25.117446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-56438049-6356-4889-b74b-514c83726036 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043417 Aug 24 10:29:25.117985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 937/1732] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:25 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:25.689561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:25.690189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:25.697353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:25.698024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:25.698652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:25.699671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:25.700333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:25.700930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:25.837505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:25.846718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:25.848265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:25.849434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b86b80f1-efd7-4354-b856-5717f00b723b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.163979 Aug 24 10:29:25.849994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 796/1733] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:25 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:29:25.860620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:25.861227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:25.867673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:25.868185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:25.868620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:25.869306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:25.869754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:25.870174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:25.888304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:25.931498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:25.939389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-05d921dc-855a-4a16-90c9-25db2702012c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082546 Aug 24 10:29:25.939986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 938/1734] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:25 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:26.130620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f1b85580-64b1-408e-9f7c-64c5c4e1b270 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:26.132822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f1b85580-64b1-408e-9f7c-64c5c4e1b270 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:26.139342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1b85580-64b1-408e-9f7c-64c5c4e1b270 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:26.139832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1b85580-64b1-408e-9f7c-64c5c4e1b270 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:26.140308 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1b85580-64b1-408e-9f7c-64c5c4e1b270 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:26.141661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1b85580-64b1-408e-9f7c-64c5c4e1b270 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:26.141661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1b85580-64b1-408e-9f7c-64c5c4e1b270 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:26.141966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1b85580-64b1-408e-9f7c-64c5c4e1b270 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:26.170752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f1b85580-64b1-408e-9f7c-64c5c4e1b270 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044105 Aug 24 10:29:26.171310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 797/1735] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:26 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:26.873059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:26.873670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:26.880161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:26.880648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:26.881085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:26.881777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:26.882234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:26.882662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:26.960660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:26.961280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:26.968639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:26.969125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:26.969562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:26.970260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:26.970715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:26.971146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:27.015650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:27.024500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:27.026013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:27.027247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8ca03d08-5965-4cff-8be8-1f1573212826 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158092 Aug 24 10:29:27.027884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 939/1736] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:26 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:27.033804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:27.041909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-25eaaab7-f125-479c-96fd-9471349fa855 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085258 Aug 24 10:29:27.042472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 798/1737] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:26 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:27.183395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-81118aa1-1330-40c1-bb2b-2ef2968c4bf7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:27.185574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-81118aa1-1330-40c1-bb2b-2ef2968c4bf7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:27.191399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81118aa1-1330-40c1-bb2b-2ef2968c4bf7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:27.191878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81118aa1-1330-40c1-bb2b-2ef2968c4bf7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:27.192341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81118aa1-1330-40c1-bb2b-2ef2968c4bf7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:27.193023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81118aa1-1330-40c1-bb2b-2ef2968c4bf7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:27.193475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81118aa1-1330-40c1-bb2b-2ef2968c4bf7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:27.193895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81118aa1-1330-40c1-bb2b-2ef2968c4bf7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:27.222413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-81118aa1-1330-40c1-bb2b-2ef2968c4bf7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043493 Aug 24 10:29:27.222949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 940/1738] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:27 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:28.050217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:28.050858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:28.057179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:28.057651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:28.058081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:28.058758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:28.059199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:28.059611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:28.061824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:28.063649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:28.068896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:28.069385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:28.069828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:28.070531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:28.071245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:28.071673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:28.133991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:28.142483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-507a1717-08e9-47f9-ac7f-bee696d13b4f tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084460 Aug 24 10:29:28.143179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 941/1739] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:28 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:28.192028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:28.201366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:28.202862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:28.204006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-385b4215-970e-4b27-9c35-693e1fb78aac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157803 Aug 24 10:29:28.204613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 799/1740] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:28 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:28.234553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-80e411b1-a3e6-4967-8234-2916d18cb26b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:28.236576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-80e411b1-a3e6-4967-8234-2916d18cb26b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:28.242583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80e411b1-a3e6-4967-8234-2916d18cb26b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:28.243072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80e411b1-a3e6-4967-8234-2916d18cb26b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:28.243509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80e411b1-a3e6-4967-8234-2916d18cb26b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:28.244233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80e411b1-a3e6-4967-8234-2916d18cb26b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:28.244692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80e411b1-a3e6-4967-8234-2916d18cb26b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:28.245121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80e411b1-a3e6-4967-8234-2916d18cb26b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:28.273607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-80e411b1-a3e6-4967-8234-2916d18cb26b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043164 Aug 24 10:29:28.274368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 942/1741] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:28 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:29.162873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:29.163441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:29.170355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:29.170838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:29.171271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:29.172103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:29.172579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:29.172874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:29.226916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:29.227524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:29.233419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:29.234911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:29.235400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:29.235841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:29.236602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:29.237057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:29.237481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:29.241828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-98ba0a9c-4a62-40de-b305-ec91a81a21ca tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082886 Aug 24 10:29:29.242409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 800/1742] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:29 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:29.286914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1542dad3-ff0d-436c-8fc8-19840f88a6d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:29.289098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1542dad3-ff0d-436c-8fc8-19840f88a6d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:29.295728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1542dad3-ff0d-436c-8fc8-19840f88a6d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:29.296252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1542dad3-ff0d-436c-8fc8-19840f88a6d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:29.296694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1542dad3-ff0d-436c-8fc8-19840f88a6d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:29.297400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1542dad3-ff0d-436c-8fc8-19840f88a6d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:29.297859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1542dad3-ff0d-436c-8fc8-19840f88a6d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:29.298285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1542dad3-ff0d-436c-8fc8-19840f88a6d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:29.328651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1542dad3-ff0d-436c-8fc8-19840f88a6d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045993 Aug 24 10:29:29.329230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 801/1743] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:29 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:29.368376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:29.376952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:29.378441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:29.379599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ee2e824d-188a-4a7f-8225-e8a0bfbd2064 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156616 Aug 24 10:29:29.380163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 943/1744] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:29:30.262489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:30.263101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:30.269442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:30.269919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:30.270351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:30.271034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:30.271486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:30.271908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:30.332889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:30.341096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d8a19928-71c2-4722-bbe3-8718a571dcba tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082490 Aug 24 10:29:30.341263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3826899c-f0f9-44c7-a520-ee6b68dad6af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:30.341687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 802/1745] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:30 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:30.343257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3826899c-f0f9-44c7-a520-ee6b68dad6af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:30.349704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3826899c-f0f9-44c7-a520-ee6b68dad6af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:30.350193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3826899c-f0f9-44c7-a520-ee6b68dad6af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:30.350637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3826899c-f0f9-44c7-a520-ee6b68dad6af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:30.351334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3826899c-f0f9-44c7-a520-ee6b68dad6af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:30.351796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3826899c-f0f9-44c7-a520-ee6b68dad6af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:30.352255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3826899c-f0f9-44c7-a520-ee6b68dad6af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:30.382946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3826899c-f0f9-44c7-a520-ee6b68dad6af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046178 Aug 24 10:29:30.383497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 944/1746] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:30 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:30.402873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:30.403475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:30.409435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:30.409906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:30.410334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:30.411008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:30.411453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:30.411869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:30.541200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:30.550150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:30.551661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:30.552860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-61703081-57cc-4f74-a7e0-0e06adef5313 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154550 Aug 24 10:29:30.553413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 803/1747] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:29:31.362050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:31.362673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:31.368987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:31.370056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:31.370499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:31.371191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:31.371648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:31.372105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:31.395783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a7b5af64-3b54-4e80-a0f7-198afe4ac16b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:31.397973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a7b5af64-3b54-4e80-a0f7-198afe4ac16b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:31.405979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a7b5af64-3b54-4e80-a0f7-198afe4ac16b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:31.406460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a7b5af64-3b54-4e80-a0f7-198afe4ac16b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:31.406887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a7b5af64-3b54-4e80-a0f7-198afe4ac16b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:31.407579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a7b5af64-3b54-4e80-a0f7-198afe4ac16b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:31.408024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a7b5af64-3b54-4e80-a0f7-198afe4ac16b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:31.408493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a7b5af64-3b54-4e80-a0f7-198afe4ac16b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:31.436903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:31.439711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a7b5af64-3b54-4e80-a0f7-198afe4ac16b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047889 Aug 24 10:29:31.440421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 804/1748] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:31 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:31.445084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-53060442-3ace-4575-b8e1-f7b116116764 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086905 Aug 24 10:29:31.445896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 945/1749] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:31 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:31.575982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:31.576541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:31.582788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:31.583265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:31.583696 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:31.584426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:31.584873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:31.585288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:31.713411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:31.722169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:31.723691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:31.725008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8bd6382f-4493-461e-b004-4658c05eb2ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153519 Aug 24 10:29:31.725576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 805/1750] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:31 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:32.452884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-58abee34-b6ff-4d9a-9f70-6602f785d91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:32.455045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-58abee34-b6ff-4d9a-9f70-6602f785d91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:32.462311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58abee34-b6ff-4d9a-9f70-6602f785d91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:32.462809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58abee34-b6ff-4d9a-9f70-6602f785d91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:32.463782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58abee34-b6ff-4d9a-9f70-6602f785d91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:32.464571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58abee34-b6ff-4d9a-9f70-6602f785d91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:32.464971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58abee34-b6ff-4d9a-9f70-6602f785d91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:32.465401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58abee34-b6ff-4d9a-9f70-6602f785d91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:32.466818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:32.467484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:32.473948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:32.474429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:32.474933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:32.475632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:32.476114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:32.476542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:32.497349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-58abee34-b6ff-4d9a-9f70-6602f785d91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048389 Aug 24 10:29:32.498148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 946/1751] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:32 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:32.541861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:32.549754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8a6a540e-3b34-4431-9544-b82b6e1c9348 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.087537 Aug 24 10:29:32.550329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 806/1752] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:32 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:32.748365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:32.749013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:32.754946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:32.755429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:32.755866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:32.756601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:32.757060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:32.757487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:32.956110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:32.965428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:32.966924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:32.968085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-daec5d1f-cc34-490b-9f1a-67bbb44350b4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.223765 Aug 24 10:29:32.968668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 947/1753] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:32 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:33.509494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bf81f996-35bb-4b0d-a835-44a882d937f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:33.512228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bf81f996-35bb-4b0d-a835-44a882d937f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:33.518749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf81f996-35bb-4b0d-a835-44a882d937f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:33.519225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf81f996-35bb-4b0d-a835-44a882d937f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:33.519657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf81f996-35bb-4b0d-a835-44a882d937f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:33.520367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf81f996-35bb-4b0d-a835-44a882d937f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:33.520814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf81f996-35bb-4b0d-a835-44a882d937f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:33.521232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf81f996-35bb-4b0d-a835-44a882d937f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:33.551494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bf81f996-35bb-4b0d-a835-44a882d937f1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045845 Aug 24 10:29:33.551970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 807/1754] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:33 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:33.570719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:33.571374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:33.578312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:33.578821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:33.579267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:33.579956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:33.580445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:33.580876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:33.642478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:33.650195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c8790fea-46e1-4279-8c3c-2bd632d91f97 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083580 Aug 24 10:29:33.651345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 948/1755] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:33 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:33.990968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:33.991572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:33.998418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:33.998898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:33.999327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:34.000023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:34.000514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:34.000915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:34.132438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:34.141218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:34.142757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:34.143947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d3cdd7fe-ec6f-4d87-a49e-a74acc6a25f2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157005 Aug 24 10:29:34.144534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 808/1756] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:34.564014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7e17e13f-971e-4138-ab01-285eed0fcb41 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:34.566193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7e17e13f-971e-4138-ab01-285eed0fcb41 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:34.572508 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e17e13f-971e-4138-ab01-285eed0fcb41 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:34.573087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e17e13f-971e-4138-ab01-285eed0fcb41 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:34.573427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e17e13f-971e-4138-ab01-285eed0fcb41 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:34.574123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e17e13f-971e-4138-ab01-285eed0fcb41 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:34.574581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e17e13f-971e-4138-ab01-285eed0fcb41 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:34.575009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e17e13f-971e-4138-ab01-285eed0fcb41 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:34.603910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7e17e13f-971e-4138-ab01-285eed0fcb41 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043948 Aug 24 10:29:34.604477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 949/1757] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:34 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:34.670960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:34.671570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:34.677946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:34.678440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:34.678871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:34.679545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:34.679990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:34.680430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:34.741467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:34.749833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5de8edcf-944b-4c7d-93f9-daf523494f28 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082641 Aug 24 10:29:34.750402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 809/1758] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:34 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:35.166782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:35.167313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:35.173309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:35.173800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:35.174244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:35.174938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:35.175398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:35.175830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:35.307294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:35.316463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:35.318238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:35.319395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-98bf7ced-f8a1-4039-b043-b09315963549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156646 Aug 24 10:29:35.319944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 950/1759] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:35 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:35.615672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6bf090ae-6504-4494-95e3-ba41af23d462 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:35.617841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6bf090ae-6504-4494-95e3-ba41af23d462 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:35.624473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6bf090ae-6504-4494-95e3-ba41af23d462 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:35.624949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6bf090ae-6504-4494-95e3-ba41af23d462 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:35.625376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6bf090ae-6504-4494-95e3-ba41af23d462 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:35.626085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6bf090ae-6504-4494-95e3-ba41af23d462 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:35.626531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6bf090ae-6504-4494-95e3-ba41af23d462 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:35.626947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6bf090ae-6504-4494-95e3-ba41af23d462 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:35.654730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6bf090ae-6504-4494-95e3-ba41af23d462 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043057 Aug 24 10:29:35.655260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 810/1760] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:35 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:35.771258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:35.771655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:35.777325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:35.777805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:35.778238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:35.778916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:35.779363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:35.779786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:35.841054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:35.848642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-62e8f561-f8e3-47df-b2fc-6b34a97bf5c7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081644 Aug 24 10:29:35.849197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 951/1761] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:35 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:36.342813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:36.343483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:36.350060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:36.350546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:36.350984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:36.351677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:36.352158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:36.352587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:36.371898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:36.491261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:36.500584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:36.502087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:36.503233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dfd72af7-1f07-4f63-af88-dda8ae65887c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.164384 Aug 24 10:29:36.503790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 811/1762] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:36 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:36.667271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-81496d86-ab5f-4848-9e17-f0fef033fc7d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:36.669368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-81496d86-ab5f-4848-9e17-f0fef033fc7d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:36.675691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81496d86-ab5f-4848-9e17-f0fef033fc7d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:36.676213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81496d86-ab5f-4848-9e17-f0fef033fc7d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:36.676652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81496d86-ab5f-4848-9e17-f0fef033fc7d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:36.677331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81496d86-ab5f-4848-9e17-f0fef033fc7d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:36.677778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81496d86-ab5f-4848-9e17-f0fef033fc7d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:36.678196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-81496d86-ab5f-4848-9e17-f0fef033fc7d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:36.707469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-81496d86-ab5f-4848-9e17-f0fef033fc7d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044698 Aug 24 10:29:36.708038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 952/1763] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:36 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:36.868767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:36.869390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:36.875344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:36.875789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:36.876258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:36.876950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:36.877405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:36.877830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:36.940923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:36.948958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c1c19c9d-5f4e-4d6a-9430-979359776259 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084119 Aug 24 10:29:36.949522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 812/1764] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:36 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:37.525765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:37.526298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:37.532320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:37.532804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:37.533243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:37.533936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:37.534394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:37.534826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:37.662166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:37.671241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:37.672766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:37.673919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-99d91eea-6afa-4918-9128-575817a30612 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152231 Aug 24 10:29:37.674466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 953/1765] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:37.719887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d9c2a525-6216-4394-8903-ef040fb1170c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:37.722095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d9c2a525-6216-4394-8903-ef040fb1170c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:37.729198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9c2a525-6216-4394-8903-ef040fb1170c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:37.729779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9c2a525-6216-4394-8903-ef040fb1170c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:37.730220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9c2a525-6216-4394-8903-ef040fb1170c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:37.730918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9c2a525-6216-4394-8903-ef040fb1170c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:37.731372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9c2a525-6216-4394-8903-ef040fb1170c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:37.731796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9c2a525-6216-4394-8903-ef040fb1170c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:37.760451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d9c2a525-6216-4394-8903-ef040fb1170c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044505 Aug 24 10:29:37.761004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 813/1766] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:37 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:37.969671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:37.970283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:37.976703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:37.977191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:37.977630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:37.978859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:37.979314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:37.979739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:38.045032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:38.052723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8985d6d7-3e00-433b-8b34-e11493fb2427 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086960 Aug 24 10:29:38.053283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 954/1767] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:37 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:38.698142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:38.698755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:38.704846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:38.705332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:38.705770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:38.706454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:38.706906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:38.707335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:38.772832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cceccfb2-e1cc-44b7-ae73-25aea9eb9577 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:38.774980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cceccfb2-e1cc-44b7-ae73-25aea9eb9577 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:38.781024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cceccfb2-e1cc-44b7-ae73-25aea9eb9577 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:38.781509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cceccfb2-e1cc-44b7-ae73-25aea9eb9577 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:38.781947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cceccfb2-e1cc-44b7-ae73-25aea9eb9577 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:38.782658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cceccfb2-e1cc-44b7-ae73-25aea9eb9577 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:38.783122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cceccfb2-e1cc-44b7-ae73-25aea9eb9577 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:38.783549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cceccfb2-e1cc-44b7-ae73-25aea9eb9577 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:38.787588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:38.815322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cceccfb2-e1cc-44b7-ae73-25aea9eb9577 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046393 Aug 24 10:29:38.815964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 955/1768] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:38 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:38.838344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:38.847091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:38.848634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:38.849791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-18dd62bc-8c5f-4f29-83b0-e9345a449b43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155818 Aug 24 10:29:38.850349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 814/1769] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:38 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:39.074236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:39.074773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:39.081215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:39.081703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:39.082150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:39.082844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:39.083328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:39.083762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:39.144446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:39.152673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-537c7a5e-0346-4af8-b476-406a2c373bcc tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082654 Aug 24 10:29:39.153239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 956/1770] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:39 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:39.826996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-187ff9a6-4a6c-4022-b419-cf9da157d28c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:39.829192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-187ff9a6-4a6c-4022-b419-cf9da157d28c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:39.835580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-187ff9a6-4a6c-4022-b419-cf9da157d28c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:39.836150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-187ff9a6-4a6c-4022-b419-cf9da157d28c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:39.836589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-187ff9a6-4a6c-4022-b419-cf9da157d28c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:39.837282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-187ff9a6-4a6c-4022-b419-cf9da157d28c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:39.838245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-187ff9a6-4a6c-4022-b419-cf9da157d28c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:39.838672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-187ff9a6-4a6c-4022-b419-cf9da157d28c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:39.869552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-187ff9a6-4a6c-4022-b419-cf9da157d28c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046414 Aug 24 10:29:39.870290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 815/1771] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:39 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:39.873458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:39.874077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:39.880007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:39.880554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:39.880996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:39.881689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:39.882146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:39.882576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:40.010401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:40.019274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:40.020889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:40.022060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bc1c1e1f-e32d-4a60-85b8-9295385dca18 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152919 Aug 24 10:29:40.022622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 957/1772] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:39 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:29:40.173932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:40.174549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:40.180530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:40.181020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:40.182250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:40.182935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:40.183384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:40.183806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:40.245971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:40.253737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8e79b089-5a07-4f0a-bca8-63afaaf40f2e tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083705 Aug 24 10:29:40.254302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 816/1773] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:40 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:40.881774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f6448e0c-7eba-425f-9e89-ad6995c26ef2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:40.883907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f6448e0c-7eba-425f-9e89-ad6995c26ef2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:40.890833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6448e0c-7eba-425f-9e89-ad6995c26ef2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:40.891305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6448e0c-7eba-425f-9e89-ad6995c26ef2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:40.891740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6448e0c-7eba-425f-9e89-ad6995c26ef2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:40.892462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6448e0c-7eba-425f-9e89-ad6995c26ef2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:40.892914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6448e0c-7eba-425f-9e89-ad6995c26ef2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:40.893337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6448e0c-7eba-425f-9e89-ad6995c26ef2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:40.922263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f6448e0c-7eba-425f-9e89-ad6995c26ef2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044569 Aug 24 10:29:40.922800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 958/1774] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:40 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:41.045030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:41.045547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:41.051937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:41.052449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:41.052877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:41.053559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:41.054002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:41.054419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:41.185339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:41.193935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:41.195682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:41.197354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7555bbcf-0519-4a17-8f24-95535f6f836c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155792 Aug 24 10:29:41.197906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 817/1775] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:41 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:41.274849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:41.275630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:41.281898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:41.282382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:41.282821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:41.283509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:41.283964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:41.284430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:41.346313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:41.354485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5a1fd257-f807-45d9-9a34-159fc9d473b2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083525 Aug 24 10:29:41.355045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 959/1776] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:41 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:41.934186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bcb96952-fca0-4636-8889-3c29eb7542e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:41.936352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bcb96952-fca0-4636-8889-3c29eb7542e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:41.942368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb96952-fca0-4636-8889-3c29eb7542e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:41.942852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb96952-fca0-4636-8889-3c29eb7542e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:41.943286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb96952-fca0-4636-8889-3c29eb7542e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:41.943975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb96952-fca0-4636-8889-3c29eb7542e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:41.944456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb96952-fca0-4636-8889-3c29eb7542e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:41.944883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcb96952-fca0-4636-8889-3c29eb7542e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:41.975382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bcb96952-fca0-4636-8889-3c29eb7542e1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045235 Aug 24 10:29:41.975933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 818/1777] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:41 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:42.219953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:42.220600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:42.226952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:42.227433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:42.227870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:42.228586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:42.229035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:42.229460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:42.360436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:42.369505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:42.371014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:42.372199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0bce33fe-f170-42e0-9672-d19943b36bad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156224 Aug 24 10:29:42.372747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 960/1778] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:42 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:42.375736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:42.377297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:42.382797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:42.383272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:42.383697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:42.384447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:42.384889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:42.385306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:42.446097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:42.454167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b7f36569-b4b6-4b0a-b14a-1b811c7f90b7 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082423 Aug 24 10:29:42.454737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 819/1779] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:42 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:42.986937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b75b642d-5a03-43c5-bd70-8f5fa3d4c0e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:42.989130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b75b642d-5a03-43c5-bd70-8f5fa3d4c0e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:42.994962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b75b642d-5a03-43c5-bd70-8f5fa3d4c0e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:42.995446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b75b642d-5a03-43c5-bd70-8f5fa3d4c0e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:42.995885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b75b642d-5a03-43c5-bd70-8f5fa3d4c0e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:42.996680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b75b642d-5a03-43c5-bd70-8f5fa3d4c0e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:42.997139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b75b642d-5a03-43c5-bd70-8f5fa3d4c0e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:42.997567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b75b642d-5a03-43c5-bd70-8f5fa3d4c0e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:43.027166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b75b642d-5a03-43c5-bd70-8f5fa3d4c0e9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044097 Aug 24 10:29:43.027643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 961/1780] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:42 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:43.395206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:43.395819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:43.402966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:43.403454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:43.403891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:43.404619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:43.405074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:43.405499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:43.474699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:43.475352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:43.481656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:43.482436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:43.482662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:43.483540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:43.483809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:43.484243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:43.534054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:43.542763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:43.544468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:43.545623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1963f32a-1af8-4a59-94e2-f598894560ce tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154413 Aug 24 10:29:43.546181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 820/1781] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:43.547490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:43.555462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9d8b4bc2-d09b-48ae-9721-f031503569a6 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084576 Aug 24 10:29:43.556453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 962/1782] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:43 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:44.038830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d0c0eda3-eef5-483b-870a-4d1d6b789c07 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:44.041060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d0c0eda3-eef5-483b-870a-4d1d6b789c07 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:44.047033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d0c0eda3-eef5-483b-870a-4d1d6b789c07 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:44.047518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d0c0eda3-eef5-483b-870a-4d1d6b789c07 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:44.047957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d0c0eda3-eef5-483b-870a-4d1d6b789c07 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:44.048684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d0c0eda3-eef5-483b-870a-4d1d6b789c07 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:44.049142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d0c0eda3-eef5-483b-870a-4d1d6b789c07 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:44.049568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d0c0eda3-eef5-483b-870a-4d1d6b789c07 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:44.081327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d0c0eda3-eef5-483b-870a-4d1d6b789c07 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046414 Aug 24 10:29:44.081898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 821/1783] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:44 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:44.568708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:44.569296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:44.576649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:44.577134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:44.577362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:44.577771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:44.578018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:44.578240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:44.578689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:44.579110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:44.584300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:44.584783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:44.585216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:44.585906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:44.586358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:44.586784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:44.651567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:44.659450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-75e60954-1516-4f0e-8b9b-dbe9734fc263 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086123 Aug 24 10:29:44.660021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 822/1784] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:44 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:44.712825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:44.721459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:44.722981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:44.724163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2feee5ea-fb3e-4596-80ce-3b92c3988a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.159358 Aug 24 10:29:44.724714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 963/1785] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:45.094222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-116f0c23-2ee4-4100-b525-f340a7c682d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:45.096379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-116f0c23-2ee4-4100-b525-f340a7c682d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:45.102504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116f0c23-2ee4-4100-b525-f340a7c682d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:45.103658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116f0c23-2ee4-4100-b525-f340a7c682d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:45.104107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116f0c23-2ee4-4100-b525-f340a7c682d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:45.104795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116f0c23-2ee4-4100-b525-f340a7c682d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:45.105249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116f0c23-2ee4-4100-b525-f340a7c682d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:45.105670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116f0c23-2ee4-4100-b525-f340a7c682d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:45.135596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-116f0c23-2ee4-4100-b525-f340a7c682d6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045418 Aug 24 10:29:45.136155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 823/1786] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:45 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:45.683152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:45.683766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:45.690144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:45.690624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:45.691062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:45.691748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:45.692260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:45.692687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:45.746790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:45.747428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:45.754043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:45.754378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:45.754541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:45.754975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:45.755670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:45.756149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:45.756574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:45.762482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3e8f7c24-8572-4bd1-82e1-2b56e3263ac5 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084364 Aug 24 10:29:45.763050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 964/1787] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:45 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:45.884379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:45.893928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:45.895843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:45.897099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-304f950c-32ec-416f-94de-8ba7ee712dfc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154223 Aug 24 10:29:45.897642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 824/1788] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:45 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:29:46.148917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f4ae08ad-416c-495e-b141-5334adef154e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:46.151694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f4ae08ad-416c-495e-b141-5334adef154e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:46.159261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4ae08ad-416c-495e-b141-5334adef154e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:46.159747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4ae08ad-416c-495e-b141-5334adef154e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:46.160218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4ae08ad-416c-495e-b141-5334adef154e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:46.160919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4ae08ad-416c-495e-b141-5334adef154e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:46.161372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4ae08ad-416c-495e-b141-5334adef154e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:46.161799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4ae08ad-416c-495e-b141-5334adef154e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:46.193155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f4ae08ad-416c-495e-b141-5334adef154e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050176 Aug 24 10:29:46.193697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 965/1789] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:46 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:46.783379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:46.783989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:46.791349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:46.791824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:46.792294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:46.792982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:46.793433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:46.793853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:46.854132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:46.861946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-61a3af83-a2df-4c5e-bed9-4c70d2a5bd43 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082422 Aug 24 10:29:46.862509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 825/1790] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:46 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:46.920253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:46.920870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:46.927299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:46.927787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:46.928268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:46.928964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:46.929425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:46.929858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:47.055835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:47.064489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:47.066001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:47.067170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-44711dcb-6bb2-47d8-a5ef-6688a8062a62 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.150863 Aug 24 10:29:47.067734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 966/1791] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:46 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:47.206046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-13c52c2c-94b1-467a-8bd6-dbddfa935835 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:47.208259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-13c52c2c-94b1-467a-8bd6-dbddfa935835 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:47.214789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c52c2c-94b1-467a-8bd6-dbddfa935835 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:47.215251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c52c2c-94b1-467a-8bd6-dbddfa935835 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:47.215683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c52c2c-94b1-467a-8bd6-dbddfa935835 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:47.216399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c52c2c-94b1-467a-8bd6-dbddfa935835 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:47.216877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c52c2c-94b1-467a-8bd6-dbddfa935835 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:47.217272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c52c2c-94b1-467a-8bd6-dbddfa935835 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:47.246086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-13c52c2c-94b1-467a-8bd6-dbddfa935835 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044014 Aug 24 10:29:47.246636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 826/1792] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:47 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:47.882505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:47.883179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:47.889799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:47.890283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:47.890724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:47.891416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:47.891875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:47.892409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:47.952719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:47.960447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f3a0eef4-2f8f-4102-878d-0edcd9abb74d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081772 Aug 24 10:29:47.961010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 967/1793] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:47 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:48.090589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:48.091109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:48.097501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:48.097976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:48.098401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:48.099077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:48.099520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:48.099956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:48.230640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:48.239422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:48.240936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:48.242109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-decb95d0-2788-40a5-a43f-a77516a926c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155474 Aug 24 10:29:48.242659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 827/1794] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:48 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:29:48.258108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5691bbbb-2c3c-434c-9b7b-0b74b5100d57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:48.260874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5691bbbb-2c3c-434c-9b7b-0b74b5100d57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:48.266778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5691bbbb-2c3c-434c-9b7b-0b74b5100d57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:48.267272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5691bbbb-2c3c-434c-9b7b-0b74b5100d57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:48.267719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5691bbbb-2c3c-434c-9b7b-0b74b5100d57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:48.268441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5691bbbb-2c3c-434c-9b7b-0b74b5100d57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:48.268903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5691bbbb-2c3c-434c-9b7b-0b74b5100d57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:48.269723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5691bbbb-2c3c-434c-9b7b-0b74b5100d57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:48.298422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5691bbbb-2c3c-434c-9b7b-0b74b5100d57 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044227 Aug 24 10:29:48.302840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 968/1795] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:48 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:48.982205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:48.983034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:48.988857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:48.989342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:48.989779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:48.990469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:48.990924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:48.991350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:48.995228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:49.051895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:49.059699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ed699eaa-088f-445a-8ed2-50baa0d3a67c tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082018 Aug 24 10:29:49.060298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 828/1796] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:48 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:49.265781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:49.266385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:49.272835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:49.273308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:49.273742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:49.275097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:49.275544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:49.275967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:49.310572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5a19ab52-febe-44af-94df-59d53f38c2be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:49.312869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5a19ab52-febe-44af-94df-59d53f38c2be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:49.318854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a19ab52-febe-44af-94df-59d53f38c2be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:49.319348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a19ab52-febe-44af-94df-59d53f38c2be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:49.320067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a19ab52-febe-44af-94df-59d53f38c2be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:49.320805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a19ab52-febe-44af-94df-59d53f38c2be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:49.321781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a19ab52-febe-44af-94df-59d53f38c2be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:49.322213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a19ab52-febe-44af-94df-59d53f38c2be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:49.352425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5a19ab52-febe-44af-94df-59d53f38c2be tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045809 Aug 24 10:29:49.352984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 829/1797] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:49 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:49.406723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:49.415241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:49.416769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:49.417919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fbbd96d2-ed12-477b-bddb-759dbfa12104 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156065 Aug 24 10:29:49.418473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 969/1798] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:50.080375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:50.080905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:50.087449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:50.087912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:50.088369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:50.089049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:50.089494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:50.089912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:50.149891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:50.157637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-91647688-fad0-4a82-9734-d02bb33e1816 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081229 Aug 24 10:29:50.158227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 830/1799] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:50 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:50.364778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7e95549e-3d96-4542-8055-97da6a31ace6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:50.366828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7e95549e-3d96-4542-8055-97da6a31ace6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:50.373358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e95549e-3d96-4542-8055-97da6a31ace6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:50.373850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e95549e-3d96-4542-8055-97da6a31ace6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:50.374304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e95549e-3d96-4542-8055-97da6a31ace6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:50.375370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e95549e-3d96-4542-8055-97da6a31ace6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:50.376834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e95549e-3d96-4542-8055-97da6a31ace6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:50.376834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e95549e-3d96-4542-8055-97da6a31ace6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:50.407836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7e95549e-3d96-4542-8055-97da6a31ace6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047128 Aug 24 10:29:50.408448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 970/1800] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:50 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:50.441190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:50.441799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:50.449325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:50.449833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:50.450266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:50.450948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:50.451397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:50.451819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:50.587827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:50.596586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:50.598100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:50.599263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-30eded2a-9c62-4a7f-89e7-d65699f6d11a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.162057 Aug 24 10:29:50.599819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 831/1801] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:50 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:51.178132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:51.178667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:51.185134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:51.185623 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:51.186063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:51.186869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:51.187328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:51.187758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:51.248065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:51.251709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:51.256757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-640c9611-3dcc-43cc-a933-2cb01e66c236 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082572 Aug 24 10:29:51.257521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 971/1802] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:51 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:51.420974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6760c7c0-f95c-4da6-bae2-78a238b35285 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:51.423373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6760c7c0-f95c-4da6-bae2-78a238b35285 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:51.429496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6760c7c0-f95c-4da6-bae2-78a238b35285 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:51.429919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6760c7c0-f95c-4da6-bae2-78a238b35285 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:51.431010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6760c7c0-f95c-4da6-bae2-78a238b35285 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:51.431707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6760c7c0-f95c-4da6-bae2-78a238b35285 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:51.432192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6760c7c0-f95c-4da6-bae2-78a238b35285 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:51.432623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6760c7c0-f95c-4da6-bae2-78a238b35285 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:51.462666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6760c7c0-f95c-4da6-bae2-78a238b35285 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044965 Aug 24 10:29:51.463436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 832/1803] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:51 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:51.623437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:51.624070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:51.630626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:51.631109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:51.631555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:51.632285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:51.632751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:51.633182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:51.762163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:51.771051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:51.772417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:51.773582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f4bf4d11-f7df-43c9-875e-36b0ee187aa2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154048 Aug 24 10:29:51.774129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 972/1804] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:52.279319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:52.279931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:52.285917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:52.286400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:52.286840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:52.287522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:52.287978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:52.288420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:52.348857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:52.356494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1ed980fc-1f32-4fe0-af54-e04a8b260718 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081757 Aug 24 10:29:52.357062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 833/1805] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:52 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:52.475182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8422b036-b13f-4105-b85a-ee056b17af8a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:52.477357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8422b036-b13f-4105-b85a-ee056b17af8a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:52.483401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8422b036-b13f-4105-b85a-ee056b17af8a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:52.483892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8422b036-b13f-4105-b85a-ee056b17af8a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:52.484371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8422b036-b13f-4105-b85a-ee056b17af8a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:52.485070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8422b036-b13f-4105-b85a-ee056b17af8a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:52.485531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8422b036-b13f-4105-b85a-ee056b17af8a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:52.485963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8422b036-b13f-4105-b85a-ee056b17af8a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:52.514868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8422b036-b13f-4105-b85a-ee056b17af8a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043717 Aug 24 10:29:52.515887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 973/1806] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:52 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:52.799188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:52.799801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:52.802133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=84 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:52.807986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:52.808501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:52.808933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:52.809615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:52.810065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:52.810486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:52.936736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:52.945628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:52.947126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:52.948317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0ad0bab0-5d53-4cf9-bb2a-fcfb09a469fc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153034 Aug 24 10:29:52.948873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 834/1807] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:52 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:53.376929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:53.377543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:53.383923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:53.384441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:53.384885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:53.385580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:53.386037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:53.386460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:53.448776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:53.456501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-241afe6a-796f-480e-8571-ab65a2e7512a tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.083492 Aug 24 10:29:53.457062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 974/1808] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:53 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:53.528030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d5130e6a-7949-4a0c-a1b0-13043a580686 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:53.530188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d5130e6a-7949-4a0c-a1b0-13043a580686 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:53.536148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5130e6a-7949-4a0c-a1b0-13043a580686 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:53.536622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5130e6a-7949-4a0c-a1b0-13043a580686 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:53.537049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5130e6a-7949-4a0c-a1b0-13043a580686 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:53.537722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5130e6a-7949-4a0c-a1b0-13043a580686 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:53.538166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5130e6a-7949-4a0c-a1b0-13043a580686 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:53.538582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5130e6a-7949-4a0c-a1b0-13043a580686 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:53.567190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d5130e6a-7949-4a0c-a1b0-13043a580686 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043715 Aug 24 10:29:53.567680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 835/1809] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:53.972278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:53.972896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:53.974729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=80 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:53.980351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:53.980838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:53.981278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:53.981968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:53.982449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:53.982876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:54.110806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:54.119959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:54.121490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:54.122649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0b38616d-8236-4427-892a-78dd9a846b91 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154343 Aug 24 10:29:54.123338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 975/1810] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:53 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:54.477905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:54.478564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:54.485094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:54.485577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:54.486009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:54.486699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:54.487150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:54.487575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:54.548037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:54.556369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c7ddc8d9-5ba9-44d6-8ab2-1e944f51361b tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082302 Aug 24 10:29:54.556937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 836/1811] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:54 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:54.579894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-11f02461-2c94-4543-9c5d-b76d769e7560 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:54.582054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-11f02461-2c94-4543-9c5d-b76d769e7560 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:54.588438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-11f02461-2c94-4543-9c5d-b76d769e7560 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:54.588920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-11f02461-2c94-4543-9c5d-b76d769e7560 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:54.589354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-11f02461-2c94-4543-9c5d-b76d769e7560 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:54.590040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-11f02461-2c94-4543-9c5d-b76d769e7560 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:54.590492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-11f02461-2c94-4543-9c5d-b76d769e7560 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:54.590915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-11f02461-2c94-4543-9c5d-b76d769e7560 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:54.620389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-11f02461-2c94-4543-9c5d-b76d769e7560 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044347 Aug 24 10:29:54.620926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 976/1812] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:54 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:55.145999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:55.146528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:55.153100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:55.153584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:55.154023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:55.154712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:55.155163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:55.155614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:55.284193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:55.292906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:55.294550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:55.295613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7c308040-e570-465b-b1be-812c900ad67d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153647 Aug 24 10:29:55.296188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 837/1813] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:55 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:55.578247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:55.578928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:55.584880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:55.585484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:55.585793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:55.586476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:55.586997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:55.587425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:55.632671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-de604d62-a58f-4f89-b3a1-d2d896845d77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:55.635491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-de604d62-a58f-4f89-b3a1-d2d896845d77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:55.641843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de604d62-a58f-4f89-b3a1-d2d896845d77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:55.642314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de604d62-a58f-4f89-b3a1-d2d896845d77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:55.642741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de604d62-a58f-4f89-b3a1-d2d896845d77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:55.643420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de604d62-a58f-4f89-b3a1-d2d896845d77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:55.643869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de604d62-a58f-4f89-b3a1-d2d896845d77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:55.644313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de604d62-a58f-4f89-b3a1-d2d896845d77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:55.652234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:55.660360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b6f7d1f3-7d68-4d7f-ad8d-f068f83acc50 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085984 Aug 24 10:29:55.661046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 977/1814] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:55 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:55.673718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-de604d62-a58f-4f89-b3a1-d2d896845d77 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044940 Aug 24 10:29:55.674264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 838/1815] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:55 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:56.318822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:56.319682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:56.326329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:56.326814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:56.327253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:56.327947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:56.328435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:56.328873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:56.460673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:56.469249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:56.470760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:56.471923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a0558d11-1273-4891-824d-4936107f3722 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157057 Aug 24 10:29:56.472591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 978/1816] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:56.681488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:56.682097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:56.686173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cbc0ac06-7a57-4919-9f1d-b3d2ac4e3eb0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:56.688355 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cbc0ac06-7a57-4919-9f1d-b3d2ac4e3eb0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:56.688704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:56.689208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:56.689616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:56.690324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:56.690772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:56.691187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:56.694927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbc0ac06-7a57-4919-9f1d-b3d2ac4e3eb0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:56.695407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbc0ac06-7a57-4919-9f1d-b3d2ac4e3eb0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:56.695840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbc0ac06-7a57-4919-9f1d-b3d2ac4e3eb0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:56.696610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbc0ac06-7a57-4919-9f1d-b3d2ac4e3eb0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:56.697068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbc0ac06-7a57-4919-9f1d-b3d2ac4e3eb0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:56.697494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbc0ac06-7a57-4919-9f1d-b3d2ac4e3eb0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:56.726087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cbc0ac06-7a57-4919-9f1d-b3d2ac4e3eb0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043712 Aug 24 10:29:56.726654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 979/1817] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:56 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:56.752073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:56.760008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cfafdd86-640f-4377-9dc4-b06f158715aa tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082917 Aug 24 10:29:56.760600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 839/1818] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:56 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:57.494846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:57.495484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:57.501659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:57.502137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:57.502573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:57.503255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:57.503708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:57.504164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:57.629734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:57.639361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:57.640896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:57.642067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bd970fd3-b720-4e12-9678-4568d1c83700 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151153 Aug 24 10:29:57.642627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 980/1819] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:57 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:57.738820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bbd6bd43-1c37-4b24-b737-536044a3bfa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:57.740987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bbd6bd43-1c37-4b24-b737-536044a3bfa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:57.748004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bbd6bd43-1c37-4b24-b737-536044a3bfa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:57.748515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bbd6bd43-1c37-4b24-b737-536044a3bfa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:57.748951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bbd6bd43-1c37-4b24-b737-536044a3bfa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:57.749644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bbd6bd43-1c37-4b24-b737-536044a3bfa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:57.750164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bbd6bd43-1c37-4b24-b737-536044a3bfa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:57.750558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bbd6bd43-1c37-4b24-b737-536044a3bfa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:57.781006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:57.781872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:57.782321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bbd6bd43-1c37-4b24-b737-536044a3bfa4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047463 Aug 24 10:29:57.783069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 840/1820] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:57 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:57.788668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:57.789182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:57.789617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:57.790318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:57.790766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:57.791192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:57.856192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:57.863851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-945cd82c-90a5-43af-8f03-e66af9f5196d tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.086730 Aug 24 10:29:57.864441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 981/1821] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:57 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:29:58.665492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:58.666098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:58.672035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:58.672540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:58.672978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:58.673659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:58.674113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:58.674538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:58.796471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-45099b58-cb6c-4d08-87bf-c4a8ca65278f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:58.799386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-45099b58-cb6c-4d08-87bf-c4a8ca65278f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:58.805847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:58.806324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45099b58-cb6c-4d08-87bf-c4a8ca65278f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:58.806448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45099b58-cb6c-4d08-87bf-c4a8ca65278f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:58.806826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45099b58-cb6c-4d08-87bf-c4a8ca65278f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:58.807523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45099b58-cb6c-4d08-87bf-c4a8ca65278f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:58.807971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45099b58-cb6c-4d08-87bf-c4a8ca65278f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:58.808416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-45099b58-cb6c-4d08-87bf-c4a8ca65278f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:58.814509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:58.816024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:58.817217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2aa56a15-6ee3-4f0e-9801-23b91b71793b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156400 Aug 24 10:29:58.817798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 841/1822] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:58 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:58.838447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-45099b58-cb6c-4d08-87bf-c4a8ca65278f tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046656 Aug 24 10:29:58.839033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 982/1823] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:58 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:29:58.885038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:58.885687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:58.891675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:58.892191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:58.892629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:58.893314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:58.893771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:58.894196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:58.955866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:58.964722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f380310f-71fb-4bbc-9ec9-0bbe2eac3cf8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084006 Aug 24 10:29:58.965286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 842/1824] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:58 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:29:59.840448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:59.841069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:59.847727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:59.848252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:59.848696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:59.849396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:59.849853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:59.850282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:59.850501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6f1437fe-f429-492a-b327-60fde6868103 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:59.853259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6f1437fe-f429-492a-b327-60fde6868103 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:59.859640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f1437fe-f429-492a-b327-60fde6868103 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:59.860230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f1437fe-f429-492a-b327-60fde6868103 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:59.860674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f1437fe-f429-492a-b327-60fde6868103 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:59.861368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f1437fe-f429-492a-b327-60fde6868103 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:59.861820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f1437fe-f429-492a-b327-60fde6868103 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:59.862243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f1437fe-f429-492a-b327-60fde6868103 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:59.892437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6f1437fe-f429-492a-b327-60fde6868103 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045726 Aug 24 10:29:59.892959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 843/1825] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:29:59 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:29:59.979678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:59.984763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:29:59.985376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:29:59.988408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:59.989917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:29:59.991087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ec5b0674-244a-4377-a8f3-51ac84cfc523 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154590 Aug 24 10:29:59.991253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:59.991643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 983/1826] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:29:59 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:29:59.991771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:59.992183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:29:59.992904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:29:59.993350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:29:59.993767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:00.053953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:00.057406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:00.062627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c7bdadd8-da25-4541-bbd0-2447fb265f39 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.081668 Aug 24 10:30:00.063190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 844/1827] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:29:59 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:30:00.905108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c0fc8863-f927-4b15-91eb-a423048307d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:00.907333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c0fc8863-f927-4b15-91eb-a423048307d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:00.913825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0fc8863-f927-4b15-91eb-a423048307d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:00.914308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0fc8863-f927-4b15-91eb-a423048307d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:00.915415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0fc8863-f927-4b15-91eb-a423048307d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:00.916151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0fc8863-f927-4b15-91eb-a423048307d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:00.916616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0fc8863-f927-4b15-91eb-a423048307d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:00.917053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0fc8863-f927-4b15-91eb-a423048307d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:00.946268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c0fc8863-f927-4b15-91eb-a423048307d0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045184 Aug 24 10:30:00.946817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 984/1828] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:30:00 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:30:01.014805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:01.015404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:01.021822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:01.022242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:01.022668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:01.023343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:01.024309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:01.024734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:01.083383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:01.084032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:01.090674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:01.091164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:01.091608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:01.092329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:01.092789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:01.093219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:01.153736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:01.161384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:01.162715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:01.164258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:01.165432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d5834ed0-c8cc-4195-9a4c-25aef7d8bd54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154674 Aug 24 10:30:01.165996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 845/1829] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:01.169750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-34c6966b-878e-4aaa-9edc-2518f4e57704 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.090265 Aug 24 10:30:01.178579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 985/1830] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:01 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:30:01.958475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0df7879a-669f-4008-8e78-576356ef699d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:01.960608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0df7879a-669f-4008-8e78-576356ef699d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:01.967246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0df7879a-669f-4008-8e78-576356ef699d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:01.967718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0df7879a-669f-4008-8e78-576356ef699d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:01.968170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0df7879a-669f-4008-8e78-576356ef699d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:01.968846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0df7879a-669f-4008-8e78-576356ef699d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:01.969290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0df7879a-669f-4008-8e78-576356ef699d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:01.969707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0df7879a-669f-4008-8e78-576356ef699d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:01.999339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0df7879a-669f-4008-8e78-576356ef699d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044778 Aug 24 10:30:01.999905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 846/1831] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:30:01 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:30:02.187591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:02.188957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:02.195043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:02.196864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:02.197004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:02.197107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:02.197487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:02.198185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:02.198632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:02.199055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:02.204421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:02.204943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:02.205385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:02.206100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:02.206703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:02.207133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:02.278483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:02.288165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1216578f-1458-4462-9998-3df9295793f8 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.097186 Aug 24 10:30:02.289120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 847/1832] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:02 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:30:02.356095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:02.366043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:02.368237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:02.369423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-78d2a879-6c04-4d11-b8e6-eed238c9fe7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.185818 Aug 24 10:30:02.370167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 986/1833] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:02 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:03.011964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-89b54c9b-e129-4186-8018-4e94b33985cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:03.014145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-89b54c9b-e129-4186-8018-4e94b33985cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:03.020099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b54c9b-e129-4186-8018-4e94b33985cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:03.020532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b54c9b-e129-4186-8018-4e94b33985cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:03.020965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b54c9b-e129-4186-8018-4e94b33985cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:03.021642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b54c9b-e129-4186-8018-4e94b33985cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:03.022089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b54c9b-e129-4186-8018-4e94b33985cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:03.022506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-89b54c9b-e129-4186-8018-4e94b33985cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:03.051754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-89b54c9b-e129-4186-8018-4e94b33985cd tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044248 Aug 24 10:30:03.052331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 848/1834] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:30:03 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:30:03.312636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:03.313732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:03.320278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:03.321476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:03.321922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:03.322632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:03.323090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:03.323520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:03.385173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:03.393118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6ec4db02-fb26-4833-87ca-3e55f474fcc0 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.084484 Aug 24 10:30:03.393603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:03.394063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 987/1835] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:03 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:03.394283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:03.400726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:03.401201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:03.401638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:03.402317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:03.402769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:03.403190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:03.530474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:03.539086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:03.541167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:03.542323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-16988cbc-6488-4b3f-a30c-66082bae8a43 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152605 Aug 24 10:30:03.542876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 849/1836] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:30:04.063864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e1304cc1-0539-4c10-8068-f5c87b26ea76 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:30:04.066024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e1304cc1-0539-4c10-8068-f5c87b26ea76 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:04.072855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1304cc1-0539-4c10-8068-f5c87b26ea76 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:04.073343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1304cc1-0539-4c10-8068-f5c87b26ea76 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:04.073780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1304cc1-0539-4c10-8068-f5c87b26ea76 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:04.074468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1304cc1-0539-4c10-8068-f5c87b26ea76 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:04.074923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1304cc1-0539-4c10-8068-f5c87b26ea76 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:04.075347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1304cc1-0539-4c10-8068-f5c87b26ea76 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:04.091902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:04.174044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e1304cc1-0539-4c10-8068-f5c87b26ea76 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/action" status: 200 len: 76260 microversion: 2.1 time: 0.113997 Aug 24 10:30:04.174890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 988/1837] 10.222.0.86 () {60 vars in 1278 bytes} [Wed Aug 24 10:30:04 2022] POST /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951/action => generated 76260 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:30:04.413557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:04.414113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:04.421023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:04.421505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:04.421944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:04.422657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:04.423109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:04.423554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:04.566521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:04.567225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:04.574662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:04.575148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:04.575732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:04.576310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:04.576765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:04.577190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:04.638325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:04.646216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-38ca0dea-5d9c-4761-ad36-6bb1c83bd021 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.236570 Aug 24 10:30:04.646774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 850/1838] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:04 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:04.706438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:04.715742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:04.717443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:04.718602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b366c124-7048-4087-9d6c-c1fe5ac81de5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156525 Aug 24 10:30:04.719151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 989/1839] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:30:05.668180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:05.668733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:05.675110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:05.675603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:05.676055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:05.676748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:05.677203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:05.677627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:05.740367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:05.743045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:05.743653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:05.748865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-67a8e6a0-9717-4dbf-b810-20c3f8f7c433 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.085220 Aug 24 10:30:05.749430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 851/1840] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:05 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:05.749975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:05.750526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:05.750969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:05.751663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:05.752138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:05.752564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:06.004962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:06.014563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:06.016096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:06.017383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-75d46a96-75d8-4dd9-afa1-7b15504ee658 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.278633 Aug 24 10:30:06.017949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 990/1841] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:30:06.770121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:06.771176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:06.777711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:06.778192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:06.778631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:06.779325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:06.779779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:06.780232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:06.893870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:06.902373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-636b8280-cd0c-4617-bd87-b181e5102d55 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.136437 Aug 24 10:30:06.902947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 852/1842] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:06 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:07.040325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:07.041060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:07.047928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:07.048358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:07.048795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:07.049484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:07.049936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:07.050360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:07.182608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:07.191970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:07.193514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:07.194937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f96e949f-859b-4286-8653-90055374ea72 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.159050 Aug 24 10:30:07.196028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 991/1843] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:07.924028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:07.924729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:07.931320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:07.931660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:07.932071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:07.932775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:07.933233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:07.933658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:08.006665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:08.015445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-63d08baf-ce63-498d-aaa1-2ba2ae134e49 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.095311 Aug 24 10:30:08.016012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 853/1844] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:07 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:08.218168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:08.218778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:08.225183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:08.225672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:08.226111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:08.227427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:08.227890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:08.228343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:08.364866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:08.373551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:08.375058 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:08.376260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4ace6094-f606-444e-9643-516494d0f1f3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.162106 Aug 24 10:30:08.376826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 992/1845] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:08 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:08.756747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:30:08.756994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:30:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:30:09.036552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:09.037192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] [instance: 0c601f9d-f26b-4095-a00c-53c422163d31] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:09.043511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:09.043991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:09.044453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:09.045138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:09.045591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:09.046018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:09.107134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c665588c50d47dbaf1f8fde082ee73a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '302c5ab2e660436e8a0af25c12ceee20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:09.114823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ae3ce915-971d-4042-b9c2-9d51d45e1212 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31" status: 200 len: 1219 microversion: 2.1 time: 0.082227 Aug 24 10:30:09.115372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 854/1846] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:09 2022] GET /compute/v2.1/servers/0c601f9d-f26b-4095-a00c-53c422163d31 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:09.283473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-af68782c-1edf-49bf-98e7-9d211c57f9d2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] Calling method '.version_select at 0x7fa33d809b80>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:09.310928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-af68782c-1edf-49bf-98e7-9d211c57f9d2 tempest-TestShelveInstance-1501500823 tempest-TestShelveInstance-1501500823-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-2092334647" status: 202 len: 0 microversion: 2.1 time: 0.031268 Aug 24 10:30:09.311600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 993/1847] 10.222.0.86 () {58 vars in 1254 bytes} [Wed Aug 24 10:30:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-2092334647 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:30:09.398858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:09.399469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:09.405538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:09.406023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:09.406455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:09.407145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:09.407591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:09.408010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:09.545248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:09.554090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:09.556364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:09.557539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-abe43fb7-b036-4c40-aa75-adca3948aabe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.162649 Aug 24 10:30:09.558335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 855/1848] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:09 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:09.751144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:09.751660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:09.758264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:09.758755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:09.759193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:09.759901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:09.760389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:09.761014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:09.791654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:30:09.939473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-879af404-70a0-4876-997e-f4233b2032a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 204 len: 0 microversion: 2.1 time: 0.191877 Aug 24 10:30:09.944628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 994/1849] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:30:09 2022] DELETE /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 0 bytes in 198 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:30:09.953825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:09.954566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:09.960695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:09.962616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:09.963317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:09.964290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:09.964954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:09.965584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:10.062625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8c787d70-ee26-4330-babb-6732e6c9e473 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] Calling method '.version_select at 0x7fa33d80fee0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:10.091426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8c787d70-ee26-4330-babb-6732e6c9e473 tempest-TestVolumeBootPattern-1762380404 tempest-TestVolumeBootPattern-1762380404-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1474637294" status: 202 len: 0 microversion: 2.1 time: 0.034774 Aug 24 10:30:10.092281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 995/1850] 10.222.0.86 () {58 vars in 1263 bytes} [Wed Aug 24 10:30:10 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1474637294 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:30:10.175262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:10.183754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ce178746-290c-4057-b98d-70a20170927e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.233780 Aug 24 10:30:10.184439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 856/1851] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:09 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:10.581583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:10.582523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:10.591493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:10.592008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:10.592546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:10.593377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:10.594167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:10.594602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:10.895072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:10.904419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:10.906182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:10.907593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d17d6189-e27a-4d84-b0ab-23e198e4e944 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.330720 Aug 24 10:30:10.924851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 996/1852] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:10 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:11.208853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:11.210340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:11.217733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:11.218231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:11.218644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:11.219561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:11.220787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:11.221395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:11.560602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:11.574739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6f154a7d-fc84-4c1f-9f15-d700eed80ac3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.371387 Aug 24 10:30:11.592038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 857/1853] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:11 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:11.934200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:11.934933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:11.941267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:11.941952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:11.942566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:11.943423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:11.944734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:11.945364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:12.346443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:12.367569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:12.370120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:12.372969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9bcee6f1-fe3f-4c64-a41b-0a9bb5cd4a50 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.442499 Aug 24 10:30:12.374286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 997/1854] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:11 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:12.623956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:12.625273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:12.633378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:12.634153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:12.635989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:12.637953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:12.639653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:12.641418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:12.988207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:13.000178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e30db375-904f-4960-abcf-be1c6b05ca5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.380206 Aug 24 10:30:13.000902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 858/1855] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:12 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:13.416064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:13.417571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:13.465000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:13.467244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.026s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:13.470837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:13.474631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:13.475391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:13.478643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:13.693102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:13.710275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:13.713421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:13.715627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5b7767ef-3f4b-4bae-9dc3-d935255f4b08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.315290 Aug 24 10:30:13.716872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 998/1856] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:14.049575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:14.052847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:14.089997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:14.093960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:14.095208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:14.131912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:14.141527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.037s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:14.151612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:14.578215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:14.587495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f89827b5-0ebc-4e8e-aaae-992e32ec7a4c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.566260 Aug 24 10:30:14.588055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 859/1857] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:14 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:14.741507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:14.743322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:14.750397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:14.751146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:14.751760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:14.752506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:14.753354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:14.753941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:15.007849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:15.016872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:15.018407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:15.019617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8f6c8b0c-3faa-429b-bebd-4de541df7c49 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.282087 Aug 24 10:30:15.020198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 999/1858] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:14 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:15.608509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:15.609208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:15.615592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:15.616092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:15.616541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:15.617886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:15.618341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:15.618768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:15.954323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:15.962983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d9918f7b-e336-46ce-b63d-3b0ef5f55660 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.358445 Aug 24 10:30:15.963734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 860/1859] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:15 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:16.042997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:16.043670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:16.050801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:16.051666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:16.053327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:16.055018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:16.055938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:16.056495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:16.204557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:16.214316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:16.216010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:16.217257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-954d4354-d07b-4951-a754-c6ef2fb9f8c6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.178194 Aug 24 10:30:16.217819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1000/1860] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:16 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:30:16.984741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:16.985367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:16.996630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:16.997134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:16.997575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:16.998290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:16.998745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:17.007073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:17.043185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:17.230384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:17.240886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:17.242205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:17.253166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0a26b7c3-fc68-466b-9507-812559f7d288 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.272727 Aug 24 10:30:17.255172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:17.256391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 861/1861] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:16 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:17.257165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:17.257919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:17.258913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:17.259592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:17.260296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:17.671976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:17.681173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:17.683115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:17.684625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7294510f-5253-4c8c-b79a-c7ad376c0a3d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.448602 Aug 24 10:30:17.686716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1001/1862] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:30:18.278674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:18.279283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:18.286165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:18.287168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:18.287632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:18.288531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:18.290234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:18.290844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:18.604697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:18.631702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ebf84c26-8341-413a-af42-1a5218a4205f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.356778 Aug 24 10:30:18.633556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 862/1863] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:18 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:18.711523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:18.712173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:18.718853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:18.719340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:18.719786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:18.720562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:18.720970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:18.722020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:18.895477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:18.904669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:18.906497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:18.907689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3a395619-8ea4-4413-811d-0d4b66b19616 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.200227 Aug 24 10:30:18.908250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1002/1864] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:18 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:30:19.654205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:19.654819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:19.662241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:19.662729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:19.663219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:19.665089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:19.665680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:19.666193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:19.901836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:19.915591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-de15d6fc-4f00-40d0-9d09-758b1414c3d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.265490 Aug 24 10:30:19.916386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 863/1865] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:19 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:19.931690 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:19.932476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:19.939873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:19.940708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:19.941904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:19.942685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:19.943405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:19.944160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:20.186589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:20.196538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:20.198025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:20.199197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d3b799e7-b40e-4551-8878-230eccee5947 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.272207 Aug 24 10:30:20.199970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1003/1866] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:30:20.937893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:20.938689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:20.946571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:20.947349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:20.947793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:20.948611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:20.949076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:20.949508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:21.093554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:21.102893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3862e9dc-c53a-409f-9c98-b969bae26e8b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.169320 Aug 24 10:30:21.103469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 864/1867] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:20 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:21.228177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:21.228790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:21.243574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:21.244073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:21.244514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:21.245213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:21.245668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:21.246089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:21.385657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:21.398716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:21.400667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:21.401843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-39a7f548-de12-4760-9933-f628625d1b4f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.179351 Aug 24 10:30:21.416664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1004/1868] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:21 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:22.124430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:22.124980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:22.131400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:22.131881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:22.132341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:22.133026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:22.133476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:22.133898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:22.259866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:22.268491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3165aec6-d7e7-4a37-8e0a-23adc6c06dd3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147960 Aug 24 10:30:22.269062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 865/1869] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:22 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:22.425219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:22.425757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:22.432668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:22.433159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:22.433689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:22.434393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:22.434853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:22.435284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:22.566937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:22.571322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:22.577352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:22.578858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:22.580015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3d16dede-3924-4a80-9591-896bcf69b58b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158813 Aug 24 10:30:22.580634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1005/1870] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:22 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:23.289929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:23.290519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:23.292833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=64 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:23.298155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:23.298652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:23.299094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:23.299778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:23.300261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:23.300684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:23.431693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:23.440418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-64434d1c-a813-4757-8bcc-523f6dab29a8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154964 Aug 24 10:30:23.441004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 866/1871] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:23 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:23.603441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:23.603970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:23.612003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:23.612524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:23.612964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:23.613692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:23.614154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:23.614588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:23.749918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:23.759330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:23.760873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:23.762047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-53fd651e-0cd5-4ee2-8c50-4195dc101e88 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.163128 Aug 24 10:30:23.762604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1006/1872] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:23 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:24.462113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:24.462707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:24.469230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:24.469710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:24.470145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:24.470830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:24.471282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:24.471705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:24.602422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:24.610546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2abe7cc5-b011-4e10-9261-cb299a45cbf8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152324 Aug 24 10:30:24.611093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 867/1873] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:24 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:24.786321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:24.786929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:24.789243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=71,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:24.794750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:24.795234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:24.795670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:24.796399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:24.796851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:24.797279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:24.928324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:24.937037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:24.938546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:24.939714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-85f77c77-555b-45d3-a282-3b7ceb0e004d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158038 Aug 24 10:30:24.940310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1007/1874] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:24 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:25.630507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:25.631029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:25.638013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:25.638495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:25.638922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:25.639602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:25.640066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:25.640484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:25.772451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:25.780607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7332adfe-71c1-4664-aa87-52583a25e653 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154077 Aug 24 10:30:25.781161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 868/1875] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:25 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:25.963326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:25.963952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:25.970429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:25.970922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:25.971368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:25.972083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:25.972552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:25.972987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:26.106736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:26.115426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:26.116954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:26.118115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3d141c25-69f8-41fe-a792-5a9ff76b0d4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158897 Aug 24 10:30:26.118669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1008/1876] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:25 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:26.801179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:26.801791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:26.808227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:26.808706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:26.809139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:26.809826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:26.810284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:26.810724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:26.937701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:26.946209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3208d76b-6d8f-4cc8-b149-c7b262fde72f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148982 Aug 24 10:30:26.946764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 869/1877] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:26 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:27.142052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:27.142629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:27.148993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:27.149495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:27.149935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:27.150628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:27.151191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:27.151638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:27.278629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:27.287319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:27.288844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:27.290006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cf6b2536-5868-4850-ae31-ead8890263a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151937 Aug 24 10:30:27.290593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1009/1878] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:27 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:27.967835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:27.968507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:27.974890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:27.975371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:27.975805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:27.976524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:27.976972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:27.977397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:28.106597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:28.114773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ec204499-1e2e-46a1-bf42-d6e8e58f1b6e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150805 Aug 24 10:30:28.115344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 870/1879] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:27 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:28.313343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:28.313891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:28.320635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:28.321130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:28.321573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:28.322811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:28.323272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:28.323746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:28.455140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:28.464399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:28.465956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:28.467128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d727f56d-b8cc-46ca-bc07-d882da9aec00 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157840 Aug 24 10:30:28.467684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1010/1880] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:28 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:29.134980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:29.135533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:29.142401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:29.143273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:29.143779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:29.144663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:29.145191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:29.145688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:29.289673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:29.298851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-30b70e13-c0a6-44dc-bded-703bda8183de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.167848 Aug 24 10:30:29.299418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 871/1881] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:29 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:29.490133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:29.490767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:29.497408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:29.497961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:29.498403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:29.499107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:29.499566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:29.499993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:29.635369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:29.644321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:29.645955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:29.647680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bd4bce44-c501-4572-aebf-46e5f7ef4d6b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.160967 Aug 24 10:30:29.655137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1011/1882] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:30.319772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:30.320455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:30.328060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:30.328767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:30.329391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:30.330384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:30.330839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:30.331452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:30.466386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:30.475814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d65efa4b-3f95-4e7d-8864-f0120bfa5abf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.160178 Aug 24 10:30:30.476454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 872/1883] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:30 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:30.669698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:30.670304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:30.677020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:30.677502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:30.677963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:30.678645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:30.679101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:30.679528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:30.814100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:30.823617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:30.825182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:30.826433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0352912e-85fd-485c-ae4d-fced447bcadc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.160749 Aug 24 10:30:30.826992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1012/1884] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:31.497390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:31.498009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:31.506373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:31.506860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:31.507357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:31.508142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:31.508608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:31.509094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:31.650578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:31.659333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e7059d19-9f9f-40d2-ab17-1c7ba91fd87b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.166298 Aug 24 10:30:31.660123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 873/1885] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:31 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:31.850298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:31.850843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:31.858324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:31.858812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:31.859255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:31.859948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:31.860476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:31.860907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:31.999461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:32.008625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:32.010271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:32.012012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c1f5e177-d707-4ac1-a0ef-973bfcecb473 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.165764 Aug 24 10:30:32.012633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1013/1886] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:31 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:32.680638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:32.681369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:32.696002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:32.696520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:32.697487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:32.698182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:32.698634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:32.699058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:32.834222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:32.843552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-700fa90a-ac9c-47a7-8969-404b79c5358d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.166773 Aug 24 10:30:32.844354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 874/1887] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:32 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:33.034891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:33.035619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:33.044664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:33.045402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:33.046076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:33.047143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:33.047847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:33.048541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:33.194800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:33.204457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:33.205972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:33.207137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6916d08b-d49b-43de-8d5c-65572ba3dc27 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.176179 Aug 24 10:30:33.207703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1014/1888] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:33.864761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:33.865472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:33.872019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:33.872668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:33.873176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:33.873942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:33.874389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:33.874806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:34.009989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:34.020881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-58e8ee55-edb0-40d1-ad78-fbe72aaa92f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.159907 Aug 24 10:30:34.021391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 875/1889] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:33 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:34.230473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:34.231084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:34.238509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:34.239047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:34.239493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:34.240304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:34.240750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:34.241182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:34.416642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:34.426351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:34.427886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:34.429143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b00c42c0-653b-485c-a89e-cc2b0666f11b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.202791 Aug 24 10:30:34.429904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1015/1890] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:34 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:35.042769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:35.043369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:35.049921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:35.050391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:35.050815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:35.051502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:35.051953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:35.052385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:35.196340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:35.204884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-447736e3-f0e2-43d4-9809-4ff0afcba5e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.166536 Aug 24 10:30:35.205452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 876/1891] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:35 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:30:35.453218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:35.453862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:35.460802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:35.461288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:35.461730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:35.462957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:35.463412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:35.463838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:35.598758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:35.607638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:35.609188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:35.610349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-07c12292-79de-4066-ada8-9e8ff1e76035 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.161109 Aug 24 10:30:35.610915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1016/1892] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:35 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:36.226144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:36.226718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:36.233569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:36.234045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:36.234475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:36.235158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:36.235602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:36.236020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:36.363552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:36.372715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f7684e4f-f54a-4068-a358-14d9ecd825f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150424 Aug 24 10:30:36.373304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 877/1893] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:36 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:36.632793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:36.633352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:36.639893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:36.640424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:36.640875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:36.641573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:36.642035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:36.642470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:36.772025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:36.781020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:36.782538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:36.783837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-be0209f1-eeb4-46b0-b7fd-b7d2de045eff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154969 Aug 24 10:30:36.784749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1017/1894] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:36 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:37.392978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:37.393592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:37.400085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:37.400567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:37.401003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:37.401722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:37.402172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:37.402594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:37.406605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:37.534668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:37.543800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aec0937e-48b7-4e73-b9ab-23cb99486a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154683 Aug 24 10:30:37.544401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 878/1895] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:37 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:37.808033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:37.808586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:37.814728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:37.815212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:37.815654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:37.816409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:37.816866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:37.817294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:37.944231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:37.952970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:37.954493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:37.955652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f97adbb0-27d6-4359-8ea8-836d44946048 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152615 Aug 24 10:30:37.956237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1018/1896] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:38.565035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:38.565578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:38.573172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:38.573647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:38.574086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:38.574780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:38.575235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:38.575668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:38.704822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:38.713826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b6d1bde8-750c-4b2f-9258-2525c3fe72ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152858 Aug 24 10:30:38.714387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 879/1897] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:38 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:38.978470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:38.979083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:38.985539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:38.986030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:38.986470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:38.987156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:38.987608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:38.988037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:39.114835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:39.124067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:39.125647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:39.126811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-09658da3-a808-4a74-9539-0c5cc731d521 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152372 Aug 24 10:30:39.127369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1019/1898] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:38 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:39.734686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:39.735294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:39.742305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:39.742783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:39.743214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:39.744421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:39.744874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:39.745296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:39.880679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:39.889799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-832e341a-1c76-484a-934a-1793c8de6113 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.159008 Aug 24 10:30:39.890369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 880/1899] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:39 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:30:40.149726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:40.150347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:40.156885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:40.157369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:40.157810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:40.158531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:40.158992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:40.159422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:40.292083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:40.300634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:40.302153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:40.303316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4a686de7-1a50-47e1-81c3-6635a297224e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157579 Aug 24 10:30:40.303883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1020/1900] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:40.909967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:40.910605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:40.917018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:40.917500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:40.917937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:40.918626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:40.919080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:40.919504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:41.046574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:41.054815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c7bea7dc-d0b7-4ffc-9e95-191e2ca651bc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148717 Aug 24 10:30:41.055376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 881/1901] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:40 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:41.328204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:41.328743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:41.334597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:41.335078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:41.335523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:41.336210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:41.336662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:41.337081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:41.466209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:41.474679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:41.476194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:41.477349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d0378691-ba40-4a5c-85e9-ba4335d4c483 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153900 Aug 24 10:30:41.478493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1021/1902] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:41 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:42.076714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:42.077285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:42.083648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:42.084162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:42.084607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:42.085300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:42.085759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:42.086188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:42.213721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:42.221847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-68a7b08d-d88a-4a4c-bf2c-a955dacb1ee3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149697 Aug 24 10:30:42.222405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 882/1903] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:42 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:42.499871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:42.500513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:42.506967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:42.507451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:42.507888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:42.508612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:42.509589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:42.510015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:42.528272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:42.641269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:42.649895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:42.651452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:42.652652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-26996e1a-f90b-490e-a59f-b3c7d960724f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156688 Aug 24 10:30:42.653217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1022/1904] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:42 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:43.242873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:43.244215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:43.250639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:43.251135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:43.251597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:43.252317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:43.252776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:43.253207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:43.385325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:43.393445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cb7af994-59c0-4a3f-8b52-bec38f1fcc6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154557 Aug 24 10:30:43.394002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 883/1905] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:43 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:43.675176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:43.675731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:43.682378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:43.682837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:43.683281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:43.683979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:43.684468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:43.684899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:43.826641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:43.835232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:43.837378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:43.838561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-00b9b318-e568-4723-8634-ef46a49c66a0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.167455 Aug 24 10:30:43.839138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1023/1906] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:44.414777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:44.415424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:44.421932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:44.422405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:44.422836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:44.423512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:44.423954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:44.424393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:44.551619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:44.560717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9b53f969-b386-418b-b7af-1c4c0d2ec149 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149858 Aug 24 10:30:44.561280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 884/1907] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:44 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:44.862453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:44.863099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:44.870124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:44.870608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:44.871046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:44.871735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:44.872224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:44.872652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:45.002394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:45.011102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:45.012739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:45.013903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-87bf4127-a4ea-45e2-9b5f-06576edfb623 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155465 Aug 24 10:30:45.014460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1024/1908] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:45.581793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:45.582405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:45.589575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:45.590055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:45.590529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:45.591269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:45.591673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:45.592129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:45.727156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:45.735498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-97cd90ac-fd10-45a0-aba4-6af2bf93cc43 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.157695 Aug 24 10:30:45.736077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 885/1909] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:45 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:46.037445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:46.038059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:46.044991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:46.045744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:46.046188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:46.046876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:46.047332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:46.047758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:46.174696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:46.183882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:46.185422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:46.186576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7a92e2d6-285c-45d3-889d-5fbbd87973b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153167 Aug 24 10:30:46.187131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1025/1910] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:46 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:46.756466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:46.757116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:46.764374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:46.764860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:46.765286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:46.765974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:46.766418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:46.766859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:46.893471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:46.902211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b37ad684-f61c-48a6-ae86-33e576b7f8bb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149625 Aug 24 10:30:46.902770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 886/1911] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:46 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:47.209241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:47.209880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:47.216286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:47.216777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:47.217188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:47.217861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:47.218304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:47.218718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:47.347149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:47.355908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:47.357438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:47.358599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-caddd169-d528-46be-990f-c866276c8e53 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153270 Aug 24 10:30:47.359158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1026/1912] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:47.923236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:47.923884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:47.932390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:47.932875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:47.933313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:47.934006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:47.934458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:47.934909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:48.062403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:48.070835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7f199d6c-7d4a-4c51-ad9b-d455c2008305 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151557 Aug 24 10:30:48.072038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 887/1913] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:47 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:48.382040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:48.382634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:48.388931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:48.389416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:48.389856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:48.390545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:48.390998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:48.391451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:48.526320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:48.535734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:48.537263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:48.538440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-59c01a34-a031-4150-be38-a8868c0d8126 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.160502 Aug 24 10:30:48.538992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1027/1914] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:48 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:49.091316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:49.091840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:49.098218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:49.098692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:49.099117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:49.099793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:49.100267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:49.100687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:49.228705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:49.236773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-64aef5c4-ed6b-4da7-a093-811ef8066144 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149393 Aug 24 10:30:49.237348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 888/1915] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:49 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:49.561552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:49.562196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:49.568864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:49.569347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:49.569788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:49.570478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:49.570937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:49.571362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:49.698135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:49.706715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:49.708272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:49.709431 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e580ab07-4d8a-4ea3-a81c-edb3e0e6fc42 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151809 Aug 24 10:30:49.709999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1028/1916] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:50.258769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:50.259395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:50.265559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:50.266032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:50.266461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:50.267132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:50.267574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:50.267991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:50.393729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:50.401943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-856a5e40-46cf-46e3-bb7a-b4e66246d907 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147894 Aug 24 10:30:50.402501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 889/1917] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:50 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:50.733107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:50.733723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:50.740212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:50.740704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:50.741143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:50.741837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:50.742294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:50.743372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:50.876644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:50.885177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:50.886670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:50.887824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6d3fef98-76e2-4440-a8c4-acb6b80be2a7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158681 Aug 24 10:30:50.888408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1029/1918] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:50 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:51.422810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:51.423543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:51.429996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:51.430484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:51.430920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:51.431698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:51.432264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:51.432595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:51.562225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:51.570484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9b7ea76a-2d51-4f98-bfcb-f061978ce809 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151625 Aug 24 10:30:51.571584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 890/1919] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:51 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:51.910711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:51.911315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:51.917845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:51.918321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:51.918754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:51.919434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:51.919882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:51.920343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:52.048196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:52.057407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:52.058927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:52.060101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5eeb6b3b-5b71-4d01-94f1-85ce5fecde7b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153246 Aug 24 10:30:52.060669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1030/1920] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:52.591298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:52.591914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:52.598419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:52.598909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:52.599347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:52.600038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:52.600514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:52.600939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:52.726317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:52.736263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5b8a0ef9-316b-4e88-ad9a-d2fb0a9a008e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148819 Aug 24 10:30:52.736843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 891/1921] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:52 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:53.083000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:53.083595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:53.090604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:53.091079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:53.091511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:53.092225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:53.092676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:53.093103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:53.218850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:53.227428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:53.228981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:53.230177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ab82a6e9-95dc-45cb-b17e-d6150f4dac36 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151075 Aug 24 10:30:53.230798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1031/1922] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:53 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:53.758604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:53.759217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:53.761557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=52 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:53.767239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:53.767723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:53.768181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:53.768879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:53.769337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:53.769758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:53.903173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:53.911374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c01e791a-1a7d-452c-8feb-627685d67726 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.157576 Aug 24 10:30:53.911937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 892/1923] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:53 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:54.254005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:54.254620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:54.261776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:54.262264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:54.262711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:54.263406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:54.263866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:54.264328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:54.396291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:54.405482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:54.406997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:54.408198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c419e4bd-bca2-41fc-9cde-9f20238e7dba tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158162 Aug 24 10:30:54.408767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1032/1924] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:54 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:54.932843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:54.933719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:54.939838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:54.940337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:54.940770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:54.941451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:54.941898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:54.942572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:55.069302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:55.077426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1b2a1fe9-3170-481a-870b-4a5f05df5961 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149125 Aug 24 10:30:55.077980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 893/1925] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:54 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:55.430753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:55.431440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:55.433983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=52 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:55.439595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:55.440105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:55.440555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:55.441250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:55.441707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:55.442396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:55.570088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:55.578668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:55.580201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:55.581357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-752dc7a6-e51c-4151-bddc-58cb3f1b25c5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154623 Aug 24 10:30:55.581916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1033/1926] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:55 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:56.097867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:56.098430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:56.104912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:56.105396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:56.105833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:56.106525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:56.106979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:56.107407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:56.234852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:56.238216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:30:56.244307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-568aea9b-054d-434e-b938-428eb2b6c823 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150334 Aug 24 10:30:56.244866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 894/1927] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:56 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:56.605014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:56.605598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:56.611928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:56.612434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:56.612869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:56.613546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:56.613996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:56.614413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:56.747877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:56.756674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:56.758268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:56.759425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-13d2f6e8-c0f5-42b1-bcf9-b47beb29e88c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158790 Aug 24 10:30:56.759990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1034/1928] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:57.265016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:57.265627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:57.271982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:57.272495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:57.272935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:57.274330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:57.274783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:57.275209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:57.403153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:57.411334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f2a63d73-2050-4867-bd62-41fa7c7de9be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150474 Aug 24 10:30:57.411894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 895/1929] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:57 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:57.782463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:57.783088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:57.789556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:57.790042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:57.790488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:57.791186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:57.791645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:57.792108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:57.918811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:57.927305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:57.928839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:57.930486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d09b2a70-83e8-40ac-9fa6-42414f180acf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151915 Aug 24 10:30:57.931200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1035/1930] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:57 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:58.432409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:58.433006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:58.439322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:58.439792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:58.440258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:58.440937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:58.441382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:58.441801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:58.568167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:58.576281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-01851be9-91a8-4814-9127-d264959b6ecf tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147805 Aug 24 10:30:58.576849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 896/1931] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:58 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:30:58.954856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:58.955458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:58.961894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:58.962376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:58.962812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:58.963500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:58.963956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:58.964439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:59.096000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:59.105170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:59.106715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:59.107875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5184eefb-6f3b-4afb-9ce1-00a67de71852 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156928 Aug 24 10:30:59.108464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1036/1932] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:30:58 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:30:59.597487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:30:59.598026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:30:59.604546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:59.605030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:59.605471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:59.606167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:30:59.606683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:30:59.607112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:30:59.738494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:30:59.746818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b36a3744-77df-42e3-825c-05add79e5986 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153311 Aug 24 10:30:59.747380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 897/1933] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:30:59 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:00.133914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:00.134590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:00.140866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:00.141347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:00.141781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:00.142473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:00.142921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:00.143342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:00.278655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:00.287199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:00.289211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:00.290360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4ec64652-23e4-495c-8964-8a78aec26e5e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.160970 Aug 24 10:31:00.291012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1037/1934] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:00 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:00.768240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:00.768859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:00.775863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:00.776366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:00.776802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:00.777492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:00.777943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:00.778362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:00.911800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:00.920773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1d83ff5d-e683-4f5e-ada6-56bdeae5857e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.156499 Aug 24 10:31:00.921336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 898/1935] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:00 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:01.313749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:01.314392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:01.321407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:01.321876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:01.322309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:01.322988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:01.323438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:01.323856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:01.450616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:01.459222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:01.460753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:01.461905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3840ebf0-190b-4989-ba02-75dec183d26f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152084 Aug 24 10:31:01.462463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1038/1936] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:01.941241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:01.942105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:01.948575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:01.949051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:01.949480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:01.950193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:01.950643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:01.951059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:02.079847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:02.088520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f638c59c-2718-495f-a727-54a56e791d82 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151121 Aug 24 10:31:02.089075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 899/1937] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:01 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:02.484920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:02.485627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:02.492451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:02.492955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:02.493398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:02.494101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:02.494563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:02.494997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:02.500239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:02.628964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:02.638228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:02.639772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:02.640963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c9b840e5-e781-4db0-b6fc-b691f86cda6c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.160013 Aug 24 10:31:02.641525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1039/1938] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:02 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:03.109825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:03.110369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:03.116784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:03.117265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:03.117701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:03.118392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:03.118841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:03.119267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:03.246070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:03.254437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5d1f9716-6674-44b2-bd27-53ebc2bc99f4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148640 Aug 24 10:31:03.254999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 900/1939] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:03 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:03.665479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:03.666022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:03.672503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:03.672989 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:03.673426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:03.674113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:03.674566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:03.674987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:03.805771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:03.814353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:03.815853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:03.817292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e1ed1b3a-1794-4323-98f0-953b61eb8a02 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156216 Aug 24 10:31:03.817846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1040/1940] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:04.277437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:04.278154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:04.284739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:04.285218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:04.285649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:04.286339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:04.286787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:04.287207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:04.413268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:04.421976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c8853a21-6824-4d78-abe0-8a0f29d70b93 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148947 Aug 24 10:31:04.422548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 901/1941] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:04 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:04.841342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:04.841965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:04.848632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:04.849116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:04.849551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:04.850237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:04.850688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:04.851114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:05.129001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:05.138757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:05.140327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:05.141546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2f160d0b-3b71-4b0e-a4b0-5e07372a1f41 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.304775 Aug 24 10:31:05.142102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1041/1942] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:05.443120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:05.443729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:05.450195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:05.450665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:05.451093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:05.451768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:05.452247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:05.452666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:05.582629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:05.591446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-be88d8a4-a744-4da9-b89a-794927d146e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152378 Aug 24 10:31:05.592417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 902/1943] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:05 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:06.165098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:06.165641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:06.171480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:06.171966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:06.172434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:06.173118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:06.173573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:06.174001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:06.310987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:06.320667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:06.322172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:06.323531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-afc6bce6-74cc-458a-932a-ec3d75b0fd3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.162554 Aug 24 10:31:06.324135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1042/1944] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:06.612816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:06.613435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:06.620470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:06.620952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:06.621389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:06.622080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:06.622537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:06.622967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:06.750488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:06.758737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aa63e138-11e5-45ee-95e7-5cc7f4eca311 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149858 Aug 24 10:31:06.759295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 903/1945] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:06 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:07.346365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:07.347012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:07.354033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:07.354541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:07.354960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:07.355642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:07.356127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:07.356551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:07.483562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:07.492135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:07.493599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:07.494754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eecf374e-6e0c-4046-bdde-1204732d418b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152344 Aug 24 10:31:07.495304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1043/1946] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:07.779675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:07.780325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:07.786808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:07.787292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:07.787726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:07.788490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:07.788944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:07.789369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:07.919059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:07.927098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e4e26032-5651-404d-a1ed-58323e43afde tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151308 Aug 24 10:31:07.927655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 904/1947] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:07 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:08.518039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:08.518687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:08.525096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:08.525582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:08.526024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:08.526717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:08.527174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:08.527605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:08.656137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:08.665318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:08.666871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:08.668030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-36706c16-e781-491d-8bed-ab628991ad37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153931 Aug 24 10:31:08.668599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1044/1948] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:08 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:08.949279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:08.949896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:08.956326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:08.956808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:08.957243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:08.957930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:08.958384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:08.958807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:09.087369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:09.095595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5b18675e-1753-46f3-97c4-4ce8ca202876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150954 Aug 24 10:31:09.096181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 905/1949] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:08 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:09.690957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:09.691821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:09.698874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:09.699348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:09.699777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:09.700486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:09.700935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:09.701354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:09.830695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:09.839308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:09.840866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:09.842041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8b52fb9d-04d0-4d2e-9859-0e38d39ff02d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154992 Aug 24 10:31:09.842594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1045/1950] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:09 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:10.116597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:10.118083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:10.124528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:10.125075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:10.125527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:10.126304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:10.126801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:10.127253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:10.253399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:10.262185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9755f8f4-2138-4738-81bb-8d504c359b91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149696 Aug 24 10:31:10.262753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 906/1951] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:10 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:10.865129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:10.865740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:10.872159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:10.872639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:10.873072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:10.873760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:10.874208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:10.874628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:11.008757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:11.017363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:11.018860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:11.020015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-aabd422d-4e47-4755-ae89-74c5338e1437 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158810 Aug 24 10:31:11.020675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1046/1952] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:10 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:11.283100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:11.283679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:11.290184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:11.290669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:11.291110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:11.291808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:11.292300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:11.292734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:11.425679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:11.434723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a238268f-4cd0-40a8-9ec0-0990720b009f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155592 Aug 24 10:31:11.435310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 907/1953] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:11 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:12.043630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:12.044219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:12.050556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:12.051039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:12.051477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:12.052264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:12.052723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:12.053151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:12.186045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:12.194946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:12.196524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:12.197683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e171ed43-af13-40ec-b1d8-3e947b1e7546 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158125 Aug 24 10:31:12.198233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1047/1954] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:12 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:12.456110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:12.456730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:12.463804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:12.464329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:12.464769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:12.465457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:12.465910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:12.466337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:12.594693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:12.604493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c983300f-104f-4b5e-b748-b387799ddd3c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152269 Aug 24 10:31:12.605075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 908/1955] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:12 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:13.221862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:13.222477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:13.229063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:13.229549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:13.229990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:13.230684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:13.231144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:13.231574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:13.361237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:13.370306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:13.371811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:13.373022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-12515cfd-c004-47d0-af8b-62182872a6db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156692 Aug 24 10:31:13.373573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1048/1956] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:13.625294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:13.625827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:13.632297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:13.632776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:13.633213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:13.633903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:13.634388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:13.634804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:13.764516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:13.772697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c2d5ee8a-8064-4782-9dbe-3d0257a5777d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151389 Aug 24 10:31:13.773255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 909/1957] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:13 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:14.396298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:14.396915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:14.404018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:14.404648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:14.405093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:14.405789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:14.406251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:14.406681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:14.536143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:14.545033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:14.546559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:14.547721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-841ffc65-2dce-482e-b7f5-08d41f23e81f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155370 Aug 24 10:31:14.548321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1049/1958] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:14 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:14.851016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:14.851635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:14.858235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:14.858719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:14.859157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:14.859857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:14.860333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:14.860761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:14.988070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:14.996946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-28a65e3d-8a33-49e6-a70a-ddc72390f346 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149757 Aug 24 10:31:14.997339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 910/1959] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:14 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:15.570655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:15.571254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:15.577698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:15.578173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:15.578603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:15.579284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:15.579732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:15.580193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:15.713978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:15.722590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:15.724176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:15.725357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8159564f-3e14-4bd0-9359-6a9de4d4c618 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158838 Aug 24 10:31:15.725906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1050/1960] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:16.017396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:16.018012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:16.025047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:16.025523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:16.025952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:16.026629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:16.027105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:16.027524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:16.045034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:16.160527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:16.168772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-26c6cab3-0d14-4613-8454-e9871a08bce9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155221 Aug 24 10:31:16.169337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 911/1961] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:16 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:16.748279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:16.748900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:16.755203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:16.755689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:16.756159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:16.756920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:16.757377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:16.757806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:16.889155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:16.898193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:16.899689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:16.900868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ec1a28ba-83f2-4a0c-a175-50e43651a349 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156641 Aug 24 10:31:16.901416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1051/1962] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:16 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:17.189068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:17.189678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:17.197204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:17.197710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:17.198150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:17.198844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:17.199300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:17.199727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:17.324981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:17.332997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a42481c7-2a05-4e31-a4dd-0ceba1d4355a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147892 Aug 24 10:31:17.333558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 912/1963] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:17 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:17.924270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:17.924878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:17.931294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:17.931774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:17.932236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:17.932924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:17.933374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:17.933825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:18.061448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:18.070963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:18.072518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:18.073673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0d09b2bf-1ed9-4f85-a5c2-403aa20d1995 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153338 Aug 24 10:31:18.074217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1052/1964] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:18.353618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:18.354162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:18.360516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:18.360996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:18.361431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:18.362111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:18.362561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:18.362986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:18.488956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:18.497868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-077a8a91-1488-4201-b846-f41f70d4fca3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148299 Aug 24 10:31:18.498434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 913/1965] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:18 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:19.097255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:19.097779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:19.104890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:19.105378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:19.105822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:19.106514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:19.106983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:19.107409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:19.238461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:19.247052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:19.248588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:19.249755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4bd1b3be-018f-4c53-8ab8-d56079a00074 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156576 Aug 24 10:31:19.250306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1053/1966] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:19.519419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:19.520035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:19.526951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:19.527436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:19.527876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:19.528597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:19.529054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:19.529479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:19.659737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:19.668019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-74591dd8-6643-4450-96d7-8c837ca4634d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152716 Aug 24 10:31:19.668616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 914/1967] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:19 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:20.272628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:20.273233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:20.280034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:20.280543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:20.280978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:20.281679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:20.282123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:20.282547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:20.409536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:20.418735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:20.420274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:20.421428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8fb2f343-1f82-4979-970c-fa577ef26c2a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152790 Aug 24 10:31:20.421977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1054/1968] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:20 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:20.688939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:20.689543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:20.697052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:20.697528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:20.697954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:20.698681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:20.699124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:20.699539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:20.831834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:20.840934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6b5acd0b-df36-4041-8847-69e213790242 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155740 Aug 24 10:31:20.841507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 915/1969] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:20 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:21.445531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:21.446140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:21.452533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:21.453015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:21.453454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:21.454136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:21.454592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:21.455018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:21.581603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:21.584996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:21.591167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:21.592702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:21.593893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-15a23ebb-ee4f-48f9-8d71-486f5ee8afe1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153264 Aug 24 10:31:21.594447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1055/1970] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:21 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:21.862279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:21.862889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:21.870171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:21.870654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:21.871090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:21.871778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:21.872281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:21.872708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:22.002141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:22.010484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b62d33a6-e69f-4e62-831a-f41e2c479c3f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152118 Aug 24 10:31:22.011052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 916/1971] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:21 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:22.618134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:22.618734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:22.625074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:22.625562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:22.626010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:22.626683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:22.627136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:22.627562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:22.755210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:22.764326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:22.765838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:22.766998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c2194099-17c4-49ee-8f4c-81da549644a8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153700 Aug 24 10:31:22.767769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1056/1972] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:22 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:23.030991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:23.031592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:23.038701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:23.039179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:23.039631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:23.040344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:23.040792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:23.041210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:23.168301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:23.176457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-988996cd-ca9c-432f-9e99-c5f1c4533d5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149540 Aug 24 10:31:23.177011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 917/1973] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:23 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:23.789802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:23.790451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:23.797048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:23.797539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:23.797984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:23.798681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:23.799141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:23.799570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:23.926903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:23.936096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:23.937649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:23.938811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2e2d044a-1987-4399-b98e-603b1673197b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153010 Aug 24 10:31:23.939361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1057/1974] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:23 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:24.197198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:24.197812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:24.200160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=52 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:24.205492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:24.205980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:24.206424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:24.207122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:24.207576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:24.208003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:24.334958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:24.343148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-97a5e0e9-ffbd-4fa9-87e7-71e4f994d7dc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149879 Aug 24 10:31:24.343709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 918/1975] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:24 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:24.963119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:24.963726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:24.970097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:24.970578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:24.971014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:24.971693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:24.972174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:24.972599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:25.105434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:25.113998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:25.115983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:25.117457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3c408e3b-5b81-47ce-ae30-0866c131b73b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158776 Aug 24 10:31:25.118019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1058/1976] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:24 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:25.364750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:25.366050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:25.372309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:25.372802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:25.373234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:25.373938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:25.374393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:25.374865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:25.504164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:25.512396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0e21d88d-e2ca-484c-b2df-4fa8a0c1e06b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151595 Aug 24 10:31:25.512964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 919/1977] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:25 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:31:26.141101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:26.141694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:26.143883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=52 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:26.149810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:26.150326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:26.150764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:26.151509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:26.151964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:26.152424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:26.280761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:26.289328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:26.290836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:26.292085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7c503cb4-abf6-440a-bc51-2063611c12f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154992 Aug 24 10:31:26.292640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1059/1978] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:26 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:26.532322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:26.532937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:26.539478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:26.539960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:26.540429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:26.541115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:26.541565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:26.541985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:26.672322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:26.681024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-19ce9a83-9b6f-4784-ab41-300e1af37284 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152630 Aug 24 10:31:26.681586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 920/1979] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:26 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:26.989843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:31:26.990637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:31:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:31:27.315019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:27.315595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:27.322283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:27.322768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:27.323213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:27.323904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:27.324390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:27.324825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:27.455296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:27.464657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:27.466162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:27.467323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ec82b49e-3853-4d32-8018-f35dd726cd44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156388 Aug 24 10:31:27.467874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1060/1980] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:27 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:27.702818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:27.703415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:27.710379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:27.710851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:27.711281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:27.711961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:27.712436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:27.712853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:27.844528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:27.852653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a0e13ec2-1223-40bb-92fb-401b65a63483 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153719 Aug 24 10:31:27.853413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 921/1981] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:27 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:28.491427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:28.492021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:28.498574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:28.499039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:28.499480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:28.500208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:28.500666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:28.501186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:28.626938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:28.635579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:28.637152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:28.638340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9e598b28-bd4c-48b9-a754-c7b05a01d0c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151463 Aug 24 10:31:28.638911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1061/1982] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:28 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:28.874040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:28.874654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:28.881052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:28.881531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:28.881967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:28.882654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:28.883107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:28.883529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:29.008626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:29.016803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d4b81b92-e704-4f69-858d-797fd2fdeaf3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.146648 Aug 24 10:31:29.017363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 922/1983] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:28 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:29.661579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:29.662145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:29.668756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:29.669241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:29.669679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:29.670368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:29.670822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:29.671250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:29.799175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:29.808413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:29.809951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:29.811110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c5c0823c-bd2a-4087-94b9-4b8744444582 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.153562 Aug 24 10:31:29.811660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1062/1984] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:31:30.038618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:30.039220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:30.045817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:30.046309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:30.046743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:30.047448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:30.047900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:30.048359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:30.181306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:30.189383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e0e9c390-d1c9-4741-9016-66d386bad4ec tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155216 Aug 24 10:31:30.189944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 923/1985] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:30 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:30.661538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:31:30.662392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:31:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:31:30.835028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:30.835647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:30.842083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:30.842572 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:30.843017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:30.843711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:30.844193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:30.844631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:30.973932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:30.983116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:30.984687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:30.985861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1494d52a-cf09-47df-8789-a50f717f17f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154919 Aug 24 10:31:30.986424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1063/1986] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:31.211451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:31.212088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:31.218838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:31.219482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:31.219960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:31.220825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:31.221323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:31.221753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:31.348521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:31.357332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f6477c9b-53a3-49fb-a345-978b03533ea9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150284 Aug 24 10:31:31.357887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 924/1987] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:31 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:32.009785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:32.010397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:32.016757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:32.017244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:32.017681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:32.018370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:32.018820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:32.019247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:32.109475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:31:32.109475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:31:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 (ip 10.222.0.86) !!! Aug 24 10:31:32.152401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:32.161120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:32.163191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:32.164372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-161aa9d1-badb-476b-8add-fcec626444db tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158581 Aug 24 10:31:32.164919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1064/1988] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:32 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:32.377884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:32.378409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:32.391616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:32.392157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:32.392707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:32.393461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:32.394093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:32.394630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:32.522417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:32.531958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3354fdae-4519-47b7-86fb-b2f48dccf1d2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.157995 Aug 24 10:31:32.532555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 925/1989] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:32 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:33.186724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:33.187346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:33.194657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:33.195137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:33.195578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:33.196302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:33.196762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:33.197193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:33.329435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:33.337941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:33.339421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:33.340602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e551f5f8-2004-40cd-8ee7-767f27cb9ddc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157820 Aug 24 10:31:33.341156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1065/1990] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:33.553516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:33.554099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:33.560444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:33.560930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:33.561365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:33.562051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:33.562519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:33.562923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:33.696572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:33.704894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-60ea33f4-74dd-4e05-8037-739f79889428 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155860 Aug 24 10:31:33.705461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 926/1991] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:33 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:34.362926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:34.363477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:34.370467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:34.370943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:34.371376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:34.372063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:34.372555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:34.372976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:34.504962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:34.513524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:34.515010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:34.516177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d01e5c27-75d2-4928-8f1c-fccd01bc0408 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157212 Aug 24 10:31:34.516728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1066/1992] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:34 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:34.726368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:34.726978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:34.733455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:34.733936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:34.734370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:34.735060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:34.735512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:34.735935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:34.870159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:34.878632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-36732468-8f61-4e4a-813e-22db6094b3c3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.156668 Aug 24 10:31:34.879195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 927/1993] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:34 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:35.540192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:35.541452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:35.547844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:35.548364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:35.548802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:35.549499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:35.549950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:35.550372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:35.794306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:35.803417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:35.804960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:35.806116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cb56e7eb-7373-4e4c-a512-4b105e55781d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.271103 Aug 24 10:31:35.806672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1067/1994] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:35 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:35.899252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:35.899853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:35.906478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:35.906957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:35.907388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:35.908104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:35.908552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:35.908969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:35.913129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:36.042400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:36.050591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cacbe212-ad6f-4d96-a6be-3a5687c07aad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155274 Aug 24 10:31:36.051161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 928/1995] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:35 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:36.829790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:36.830432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:36.837578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:36.838069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:36.838517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:36.839224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:36.839679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:36.840145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:36.966255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:36.975474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:36.977005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:36.978148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dbe7aeee-bd18-4a6f-bd20-3d17fc6cfa55 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152336 Aug 24 10:31:36.978692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1068/1996] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:36 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:37.072329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:37.072841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:37.079133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:37.079607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:37.080036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:37.080740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:37.081193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:37.081609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:37.212885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:37.221236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-653a9a99-0554-4fd6-92ad-7d438862a682 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152883 Aug 24 10:31:37.222037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 929/1997] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:37 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:38.001611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:38.002170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:38.009104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:38.009578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:38.010043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:38.010716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:38.011171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:38.011594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:38.141452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:38.150660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:38.152213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:38.153379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a90fb3fd-c1dc-4ab2-a412-e25655d66cec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155751 Aug 24 10:31:38.153932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1069/1998] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:38.243037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:38.243775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:38.250342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:38.250828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:38.251273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:38.252011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:38.252432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:38.252859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:38.378599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:38.387288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-acc13d33-7000-4a7f-b5b1-7f8439e13b17 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148751 Aug 24 10:31:38.387858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 930/1999] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:38 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:39.176202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:39.176751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:39.183087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:39.183587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:39.183991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:39.184699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:39.185149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:39.185571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:39.312289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:39.321720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:39.323205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:39.324373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6f1b6f27-3fa5-49a1-90bc-c8ffe26bebab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152169 Aug 24 10:31:39.324926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1070/2000] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:39 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:39.408670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:39.409288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:39.415673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:39.416193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:39.416628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:39.417318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:39.417772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:39.418197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:39.547747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:39.556390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9056603c-18ee-438a-8f9a-c79d6ed02ee7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151554 Aug 24 10:31:39.556944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 931/2001] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:39 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:40.348097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:40.348741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:40.355113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:40.355595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:40.356030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:40.356738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:40.357216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:40.357643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:40.484732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:40.494167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:40.495659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:40.496842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fc631861-2a13-478e-b985-56ac417bc6bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152831 Aug 24 10:31:40.497398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1071/2002] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:40.577850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:40.578455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:40.584959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:40.585446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:40.585885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:40.586574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:40.587032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:40.587460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:40.715022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:40.723063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-968afc2f-9f1f-435e-acdb-65eb03bf9f58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149107 Aug 24 10:31:40.724143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 932/2003] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:40 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:41.519896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:41.521108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:41.527530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:41.528020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:41.528501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:41.529193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:41.529651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:41.530082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:41.547747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:41.661090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:41.670785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:41.672895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:41.674066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cc301032-c03c-4df0-a8b2-4d12edab8d4b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158081 Aug 24 10:31:41.674622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1072/2004] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:41 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:41.745248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:41.745863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:41.752255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:41.752742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:41.753182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:41.753878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:41.754335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:41.755403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:41.890906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:41.899241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6563a919-e63a-42c0-a473-4667132dad4e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.157888 Aug 24 10:31:41.899802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 933/2005] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:41 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:42.697707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:42.698324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:42.705010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:42.705502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:42.705949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:42.706647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:42.707137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:42.707568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:42.839625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:42.848550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:42.850068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:42.851268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b612fc25-21e5-47e5-915f-d60d2ba4d6e8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157570 Aug 24 10:31:42.851843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1073/2006] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:42 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:42.919663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:42.920324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:42.926721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:42.927194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:42.927631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:42.928346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:42.928792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:42.929210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:43.057727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:43.066168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-56bca94d-e7ae-446f-b27c-5c2175b64184 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150364 Aug 24 10:31:43.066735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 934/2007] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:42 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:43.874264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:43.874876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:43.881381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:43.881860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:43.882304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:43.882991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:43.883448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:43.883879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:44.013781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:44.023056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:44.024536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:44.025697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-78fc2d8e-e4ad-4e3b-b61b-a977064a2fc7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155374 Aug 24 10:31:44.026255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1074/2008] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:44.087848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:44.088487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:44.094890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:44.095373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:44.095812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:44.096538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:44.096994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:44.097420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:44.226847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:44.235143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-432d9099-a046-480b-9ed8-3c7df568c98b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151432 Aug 24 10:31:44.235709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 935/2009] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:44 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:45.049283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:45.049868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:45.056321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:45.056811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:45.057255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:45.057949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:45.058442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:45.058872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:45.184251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:45.193621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:45.195179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:45.196372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ca4ea2f0-2db4-4f51-87b8-a0ba16d2a8f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151029 Aug 24 10:31:45.196930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1075/2010] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:45 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:45.256003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:45.256687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:45.263058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:45.263539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:45.263977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:45.264698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:45.265155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:45.265580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:45.392912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:45.402664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7715ba92-2a32-4024-a722-2304f0eb7e14 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151070 Aug 24 10:31:45.403223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 936/2011] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:45 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:46.219285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:46.219933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:46.226584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:46.227066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:46.227511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:46.228233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:46.228696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:46.229126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:46.358313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:46.367367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:46.368895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:46.370037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7578d7f6-89ce-4dac-84ee-3d8ae2f74f9c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154682 Aug 24 10:31:46.370580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1076/2012] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:46 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:46.423580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:46.424229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:46.431102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:46.431583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:46.432020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:46.432738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:46.433191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:46.433615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:46.561188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:46.569613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-eb957be1-b54f-451e-bc64-43b78fd0d16a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149950 Aug 24 10:31:46.570185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 937/2013] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:46 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:47.393191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:47.393799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:47.400128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:47.400609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:47.401043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:47.401723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:47.402173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:47.402595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:47.529456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:47.538509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:47.539999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:47.541177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5571d9c8-fd0d-48a0-a2eb-7d1251d3c757 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151898 Aug 24 10:31:47.541724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1077/2014] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:47.590672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:47.591463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:47.598757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:47.599179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:47.599608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:47.600329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:47.600776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:47.601193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:47.729158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:47.737283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e907d7df-99c8-48ec-ac75-9ec351a84268 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150702 Aug 24 10:31:47.737844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 938/2015] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:47 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:48.564261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:48.565151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:48.572133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:48.572614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:48.573050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:48.573733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:48.574186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:48.574611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:48.699654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:48.708488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:48.710140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:48.711930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-19d190c3-18fe-4886-80ab-abd5631947c0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151588 Aug 24 10:31:48.712901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1078/2016] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:48 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:48.757953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:48.758564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:48.765536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:48.766024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:48.766449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:48.767142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:48.767590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:48.768011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:48.895720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:48.903924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-46343fd7-734d-44e5-8095-874af20796a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149850 Aug 24 10:31:48.904515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 939/2017] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:48 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:49.735231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:49.735865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:49.743006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:49.743537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:49.743982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:49.744732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:49.745218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:49.745649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:49.879166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:49.887786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:49.889319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:49.890482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b3228967-0e8c-435f-b0aa-6bec37bc55cb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.159189 Aug 24 10:31:49.891040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1079/2018] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:49.924908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:49.925612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:49.931831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:49.932346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:49.932785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:49.933474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:49.933929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:49.934353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:50.072814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:50.076710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bcfe9c5e-bd67-4098-a321-5f0ab03f1617 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155634 Aug 24 10:31:50.077280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 940/2019] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:49 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:50.914606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:50.915191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:50.921466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:50.921937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:50.922369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:50.923042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:50.923490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:50.923910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:51.049807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:51.058667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:51.060210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:51.061407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9296fb16-54e1-4871-9b93-9787537e2716 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151227 Aug 24 10:31:51.061968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1080/2020] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:50 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:51.098118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:51.098837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:51.105231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:51.105721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:51.106158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:51.106845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:51.107298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:51.107721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:51.233349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:51.241696 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0686d7f2-308b-4551-8b64-d6447c22c073 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147467 Aug 24 10:31:51.242262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 941/2021] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:51 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:52.085804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:52.086384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:52.092915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:52.093406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:52.093846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:52.094534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:52.094988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:52.095415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:52.227394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:52.235937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:52.237464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:52.238659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2cb43e5b-7c81-40ae-8b23-a578cf41e2e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.157497 Aug 24 10:31:52.239201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1081/2022] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:52 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:52.263766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:52.264383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:52.275943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:52.276691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:52.277121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:52.277820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:52.278265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:52.278684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:52.405146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:52.413944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1d620c6e-fff5-4818-975f-957826774c6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154160 Aug 24 10:31:52.414514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 942/2023] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:52 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:53.261316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:53.261924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:53.268358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:53.268833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:53.269266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:53.270618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:53.271071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:53.271489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:53.397727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:53.406528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:53.408026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:53.409194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2deb2ce1-b642-4a2e-89d2-c91f466cc586 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151781 Aug 24 10:31:53.409740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1082/2024] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:53 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:31:53.434359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:53.434971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:53.442218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:53.442773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:53.443207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:53.443899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:53.444376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:53.444797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:53.575228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:53.583901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6a829eb4-30eb-4aa1-8cc2-860563ef0876 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153481 Aug 24 10:31:53.584493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 943/2025] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:53 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:54.432550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:54.433178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:54.439859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:54.440374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:54.440819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:54.442160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:54.442618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:54.443050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:54.571544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:54.580201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:54.581701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:54.583344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4b03e423-494c-4831-a01b-77598cc41362 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154770 Aug 24 10:31:54.583890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1083/2026] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:54 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:54.604602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:54.605220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:54.607534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=52 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:54.612904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:54.613452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:54.613900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:54.614770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:54.615042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:54.615468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:54.742428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:54.745776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:54.751372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f2dc7043-5efc-43aa-8daf-d4cdb0e3461e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150606 Aug 24 10:31:54.751930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 944/2027] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:54 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:55.606444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:55.607092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:55.614267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:55.614754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:55.615195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:55.615885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:55.616376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:55.616825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:55.747767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:55.756653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:55.758208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:55.759371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a2a15b89-9447-49a9-a91a-1fabd0fee6f0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156905 Aug 24 10:31:55.759926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1084/2028] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:55 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:55.776037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:55.776721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:55.784640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:55.785629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:55.786064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:55.786766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:55.787212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:55.787634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:55.917382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:55.925654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6467eb67-6d15-4d7c-82a1-3e515395f992 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155088 Aug 24 10:31:55.926213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 945/2029] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:55 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:56.782459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:56.783074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:56.785446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=52 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:31:56.790647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:56.791128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:56.791564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:56.792294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:56.792746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:56.793168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:56.921131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:56.930262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:56.931783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:56.932961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3b75e2e8-05b1-4444-b08b-5962a3a1b553 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.154446 Aug 24 10:31:56.933516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1085/2030] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:56.947426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:56.948173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:56.954052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:56.954526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:56.954955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:56.955631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:56.956111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:56.956532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:57.081974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:57.090038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-575a02eb-63d2-4924-a6b7-4370a63f96c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.146997 Aug 24 10:31:57.090627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 946/2031] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:56 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:31:57.957238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:57.957860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:57.964300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:57.964789 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:57.965230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:57.965920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:57.966378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:57.966803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:58.099431 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:58.108203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:58.109725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:58.110667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:58.111145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c850c84e-d860-4679-9060-74758216ab33 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158214 Aug 24 10:31:58.111261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:58.111431 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1086/2032] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:57 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:58.118434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:58.118912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:58.119344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:58.120054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:58.120512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:58.120936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:58.247506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:58.256250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b4f548ba-da72-40f7-8002-332a972b6ac2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149339 Aug 24 10:31:58.256825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 947/2033] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:58 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:31:59.134377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:59.134940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:59.141358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:59.141841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:59.142281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:59.142957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:59.143411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:59.143835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:59.270250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:59.277084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:31:59.277692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:31:59.278923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:59.280459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:59.281617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-90597b34-be74-475f-81c3-c69b6e1076f1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.151235 Aug 24 10:31:59.282172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1087/2034] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:31:59 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:31:59.284025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:59.284527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:59.284958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:59.285641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:31:59.286154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:31:59.286568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:31:59.411932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:31:59.419972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6c38c121-9af4-42d5-aa7b-d5b5f0788fa5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147014 Aug 24 10:31:59.420559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 948/2035] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:31:59 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:00.305426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:00.306017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:00.312397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:00.312884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:00.313327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:00.314038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:00.314752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:00.315180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:00.441306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:00.441903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:00.445244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:00.448372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:00.448853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:00.449280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:00.449967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:00.450412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:00.450832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:00.453845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:00.455347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:00.456585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-73dc1995-69f4-4b2f-9ebc-2c03b4846a61 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.155101 Aug 24 10:32:00.457147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1088/2036] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:00 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:00.577710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:00.585840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-25fdb246-237a-468a-832a-6dde4221d77d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149196 Aug 24 10:32:00.586393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 949/2037] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:00 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:01.479239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:01.479932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:01.486259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:01.486738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:01.487177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:01.488470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:01.488922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:01.489351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:01.493325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:01.606405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:01.607020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:01.614162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:01.614644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:01.615079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:01.615774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:01.616258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:01.616677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:01.616811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:01.625288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:01.626782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:01.627925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-36d67335-504f-4001-99f6-c6a2f514bf37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.152600 Aug 24 10:32:01.628499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1089/2038] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:01.746156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:01.754356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bb72ee7e-7ed3-4a87-8e02-8ae74201f0da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151850 Aug 24 10:32:01.755024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 950/2039] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:01 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:02.650932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:02.651525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:02.657919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:02.658402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:02.658836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:02.659537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:02.659993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:02.660443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:02.778778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:02.780213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:02.787566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:02.788177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:02.788839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:02.789919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:02.790663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:02.791381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:02.796812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:02.806514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:02.808061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:02.809401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-958f409a-a88d-4278-94bf-14ba86cd8c4a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.162231 Aug 24 10:32:02.820107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1090/2040] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:02 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:02.932320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:02.941496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ba8047ff-e085-46b0-8552-3c2664cbb4ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.165941 Aug 24 10:32:02.942079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 951/2041] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:02 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:03.834067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:03.834711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:03.841141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:03.841620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:03.842055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:03.842735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:03.843186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:03.843610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:03.963329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:03.963971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:03.970353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:03.970835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:03.971266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:03.971978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:03.972460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:03.972885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:03.976416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:03.985306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:03.986811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:03.987967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-86b45ac9-7aab-46de-9c4a-c9679dd3f25f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158485 Aug 24 10:32:03.988584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1091/2042] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:04.106729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:04.115324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-36d7c323-ae9c-4441-bbd1-bbef4f607ead tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155929 Aug 24 10:32:04.115878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 952/2043] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:03 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:05.011326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:05.011870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:05.018302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:05.019286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:05.019727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:05.020437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:05.020894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:05.021321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:05.136844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:05.137450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:05.144322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:05.144811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:05.145316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:05.146055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:05.146511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:05.146932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:05.152002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:05.161366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:05.162909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:05.164114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c210fdda-99d4-4eb6-98cb-e54e74fb18d8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.156712 Aug 24 10:32:05.164676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1092/2044] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:05.274475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:05.282619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f10553f7-ce4e-4c9b-8604-5f480e6a21ff tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149671 Aug 24 10:32:05.283178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 953/2045] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:05 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:06.187272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:06.187887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:06.194976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:06.195466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:06.195906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:06.196699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:06.197154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:06.197579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:06.303958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:06.306063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:06.316228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:06.316676 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:06.317102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:06.317801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:06.318245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:06.318662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:06.330050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:06.339414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:06.340963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:06.342125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0e2527ac-362e-49ef-b72f-985abe2d6465 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.158777 Aug 24 10:32:06.342689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1093/2046] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:06.449500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:06.458290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-99b0ac2f-4efe-4104-86ee-c1609e8361e7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.159402 Aug 24 10:32:06.458851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 954/2047] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:06 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:07.365729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:07.366342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:07.373435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:07.373929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:07.374369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:07.375056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:07.375514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:07.375943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:07.478849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:07.479535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:07.486471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:07.487030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:07.488423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:07.489653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:07.490211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:07.490821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:07.514079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:07.523758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:07.525409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:07.526652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c665e0e4-1403-4df4-b20b-b4a361356e81 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.165500 Aug 24 10:32:07.527888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1094/2048] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:07.627072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:07.635292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e1655617-4706-4328-a36d-0c5664e51b8c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.160381 Aug 24 10:32:07.635850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 955/2049] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:07 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:08.021924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:08.022417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:08.029462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:08.030466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:08.030792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:08.031474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:08.031924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:08.032378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:08.057457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:32:08.142176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-62569c81-d25a-41de-8802-4bd381310e43 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 204 len: 0 microversion: 2.1 time: 0.125144 Aug 24 10:32:08.142746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1095/2050] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:32:08 2022] DELETE /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:32:08.151295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:08.151914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:08.157968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:08.158448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:08.159510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:08.160549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:08.160837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:08.161445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:08.298350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:08.315696 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4cc30ddd-3423-4c5b-982c-a30677f8d96f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.166904 Aug 24 10:32:08.316300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 956/2051] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:08 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:08.550967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:08.551590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:08.557947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:08.558465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:08.558910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:08.559601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:08.560079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:08.560516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:08.658053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:08.658668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:08.665403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:08.665893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:08.666330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:08.667025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:08.667480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:08.667907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:08.721721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:08.739224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:08.740800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:08.742114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a590843a-ce3f-4887-9219-87730e8ca430 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1803 microversion: 2.20 time: 0.195908 Aug 24 10:32:08.748095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1096/2052] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:08 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1803 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:09.055373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:09.063864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e7f20cd8-d4f9-4ce2-bd88-c2c333a6e06c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.409719 Aug 24 10:32:09.064739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 957/2053] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:08 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:09.342953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:09.343599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:09.350271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:09.350693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:09.351111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:09.351803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:09.352375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:09.352809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:09.561075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:09.570077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0005e166-9008-4d44-a781-604e0bec303a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.230872 Aug 24 10:32:09.570674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1097/2054] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:09 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:09.756185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-96f8f764-1784-422f-a507-8d9d436bb8d0 req-8c373018-e45e-487d-9b72-10d3b9b892ab service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1", "name": "network-vif-deleted", "tag": "1b9069bd-2630-4b65-9e91-d7695e3b6a8c"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:32:09.767663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-96f8f764-1784-422f-a507-8d9d436bb8d0 req-8c373018-e45e-487d-9b72-10d3b9b892ab service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:09.769087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-96f8f764-1784-422f-a507-8d9d436bb8d0 req-8c373018-e45e-487d-9b72-10d3b9b892ab service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:09.769703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-96f8f764-1784-422f-a507-8d9d436bb8d0 req-8c373018-e45e-487d-9b72-10d3b9b892ab service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:09.774427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:09.776199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:09.786295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:09.786917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:09.787643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-96f8f764-1784-422f-a507-8d9d436bb8d0 req-8c373018-e45e-487d-9b72-10d3b9b892ab service nova] Creating event network-vif-deleted:1b9069bd-2630-4b65-9e91-d7695e3b6a8c for instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 on hv-cinder-85427 Aug 24 10:32:09.791304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:09.792090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:09.792780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:09.793414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:09.795339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-96f8f764-1784-422f-a507-8d9d436bb8d0 req-8c373018-e45e-487d-9b72-10d3b9b892ab service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.181739 Aug 24 10:32:09.798624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 958/2055] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:32:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:32:09.862451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:09.871500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:09.873035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:09.874175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b010f2de-0cc4-40f3-9fe0-f61ab8ee74b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.111837 Aug 24 10:32:09.874733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1098/2056] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:09 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:10.085353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:10.086121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:10.092294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:10.092776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:10.093214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:10.093910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:10.094359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:10.094781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:10.240984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:10.249048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8eb28d5d-6c9c-46b2-8fd3-c9e0541c58ca tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.167960 Aug 24 10:32:10.249602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 959/2057] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:10 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:10.592658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:10.593283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:10.600619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:10.601108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:10.601547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:10.602241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:10.602696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:10.603120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:10.741064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:10.749579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-952739a0-88c7-4fea-a1dd-c8f4fb45ecb9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.161141 Aug 24 10:32:10.750148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1099/2058] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:10 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:10.896684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:10.897296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:10.903819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:10.904297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:10.904737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:10.905424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:10.905878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:10.906305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:10.968633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:10.977910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:10.979446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:10.980618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-81cb2e17-89ec-4166-8de4-2e4be5b1f122 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088397 Aug 24 10:32:10.981193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 960/2059] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:10 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:11.270584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:11.271227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:11.277373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:11.277904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:11.278348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:11.279041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:11.279503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:11.279937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:11.409656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:11.417861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7bfe3ca2-01ed-498b-8f20-e6f7bd5c25db tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151771 Aug 24 10:32:11.418418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1100/2060] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:11 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:11.771542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:11.771682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:11.777534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:11.778017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:11.778460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:11.779150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:11.779609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:11.780035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:11.921028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:11.929766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3188cc87-7dea-4362-9e6c-067d81206e77 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.162855 Aug 24 10:32:11.930330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 961/2061] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:11 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:12.002814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:12.003430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:12.009813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:12.010299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:12.010746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:12.011435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:12.011896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:12.012351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:12.073496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:12.082295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:12.083810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:12.084969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-77af22d0-fdee-4bc6-a884-4d34a76be4ee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.086098 Aug 24 10:32:12.085529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1101/2062] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:11 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:12.438571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:12.439442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:12.445401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:12.445876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:12.446307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:12.446986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:12.447432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:12.447849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:12.451875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:12.574005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:12.582566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-95a6e88c-3710-4dd6-a908-46ad5b9bd37e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147878 Aug 24 10:32:12.583125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 962/2063] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:12 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:12.951105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:12.951733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:12.959596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:12.960124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:12.960573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:12.961293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:12.961758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:12.962189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:13.089941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:13.098236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-464f55a6-2831-4d53-9f3a-b8e25290068f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151319 Aug 24 10:32:13.098803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1102/2064] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:12 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:13.107110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:13.107857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:13.115066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:13.115551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:13.115991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:13.116722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:13.117695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:13.118133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:13.182278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:13.191837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:13.193415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:13.194568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6df7c501-b293-4516-945f-3cf8960d4958 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091414 Aug 24 10:32:13.195132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 963/2065] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:13.603527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:13.604196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:13.610270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:13.610758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:13.611201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:13.612574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:13.613024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:13.613449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:13.742190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:13.750305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-39858dba-c0b8-476f-945e-e43ac59313c9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150641 Aug 24 10:32:13.750862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1103/2066] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:13 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:14.121018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:14.121559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:14.128912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:14.129397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:14.129830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:14.130520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:14.130976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:14.131404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:14.216913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:14.218126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:14.225381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:14.225871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:14.226312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:14.227005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:14.227462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:14.227888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:14.232009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:14.262814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:14.271295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-39e5288d-bbe7-4252-8802-a558e07a13ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154917 Aug 24 10:32:14.271912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 964/2067] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:14 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:14.291262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:14.300577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:14.302146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:14.303295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bddfae60-658a-4a4c-ba0f-f69dbbf2378e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090310 Aug 24 10:32:14.303857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1104/2068] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:14 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:14.771642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:14.771888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:14.777821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:14.778302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:14.778735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:14.779420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:14.779872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:14.780328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:14.907674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:14.916417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fe8babf7-86d5-4d24-b0e7-8ff74f2afd3e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149501 Aug 24 10:32:14.916979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 965/2069] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:14 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:15.292266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:15.292865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:15.299680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:15.300191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:15.300636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:15.301314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:15.301761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:15.302185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:15.325725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:15.326364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:15.332961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:15.333449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:15.333885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:15.334576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:15.335027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:15.335449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:15.401455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:15.410629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:15.412187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:15.413326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f599b7d0-4434-4257-b3f1-6b72c6ba9645 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091506 Aug 24 10:32:15.414071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 966/2070] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:15.431662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:15.440815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fe1639bd-d668-4016-b585-a64aca38085f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152452 Aug 24 10:32:15.441371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1105/2071] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:15 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:15.937195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:15.937805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:15.943991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:15.944509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:15.944942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:15.945650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:15.946101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:15.946526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:16.072579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:16.081132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1c0d385c-9eed-4265-9302-5b60dccf2c33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147794 Aug 24 10:32:16.081738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 967/2072] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:15 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:16.435032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:16.435650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:16.443384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:16.443870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:16.444344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:16.445050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:16.445493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:16.445924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:16.460760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:16.461372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:16.467957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:16.468478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:16.468917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:16.469652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:16.470420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:16.471523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:16.509979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:16.519001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:16.520707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:16.521863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ef67024b-5130-44d6-9a22-1d7760268de9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090838 Aug 24 10:32:16.522425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1106/2073] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:16 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:16.603466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:16.612394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d873f5df-ab9b-4265-875a-3a76c25f55e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154903 Aug 24 10:32:16.612968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 968/2074] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:16 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:17.101990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:17.102521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:17.108434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:17.108924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:17.109361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:17.110043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:17.110499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:17.110925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:17.238666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:17.246746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-91e1817c-5f34-4723-816b-77c649aff37a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148766 Aug 24 10:32:17.247295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1107/2075] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:17 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:17.544745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:17.545345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:17.552285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:17.552757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:17.553182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:17.553858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:17.554302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:17.554717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:17.620587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:17.629100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:17.630596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:17.631720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-185b0987-e629-4d2e-a5f1-ef4fa9cfcd7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090867 Aug 24 10:32:17.632300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 969/2076] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:17.632687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:17.633305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:17.639750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:17.640204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:17.640643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:17.641336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:17.641790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:17.642214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:17.769155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:17.777401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-35f9d1fe-cd39-499b-9428-186388ce447b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148611 Aug 24 10:32:17.777958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1108/2077] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:17 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:18.267379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:18.268594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:18.274563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:18.275049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:18.275489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:18.276204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:18.276660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:18.277086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:18.401830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:18.410023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bc42b51b-5c2a-42b4-924e-b63e10165899 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.146570 Aug 24 10:32:18.410587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 970/2078] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:18 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:18.654470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:18.655024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:18.661456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:18.661943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:18.662383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:18.663070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:18.663529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:18.663959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:18.727847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:18.743677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:18.745193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:18.746325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-678fd2a0-9c31-4233-9f05-e078ad86cd8f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.095863 Aug 24 10:32:18.746874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1109/2079] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:18 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:18.798616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:18.799826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:18.806121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:18.806595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:18.807025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:18.807703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:18.808210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:18.808626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:18.934958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:18.943485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9dd81b72-9783-4fb0-81aa-3039dc7f4884 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148691 Aug 24 10:32:18.944082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 971/2080] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:18 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:19.430765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:19.431406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:19.437359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:19.437851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:19.438296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:19.439250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:19.440239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:19.440672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:19.566262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:19.574538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fd731656-548c-4083-9f13-af790ccd7e2e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147695 Aug 24 10:32:19.575101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1110/2081] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:19 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:19.768546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:19.769159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:19.775125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:19.775604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:19.776032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:19.776733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:19.777178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:19.777596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:19.841464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:19.850581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:19.852094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:19.853220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-169e7d68-01f4-4a34-9f5e-6f52463aea30 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088777 Aug 24 10:32:19.853772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 972/2082] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:19.973247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:19.973862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:19.980563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:19.981054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:19.981502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:19.982187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:19.982646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:19.983073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:20.110748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:20.119415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7fbe94da-8556-420d-944f-a16a7c42c8f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150660 Aug 24 10:32:20.119980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1111/2083] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:19 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:20.596627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:20.597229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:20.603123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:20.603599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:20.604026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:20.604727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:20.605173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:20.605590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:20.733342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:20.741610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-78977d8d-bd02-4a15-8373-a7d0755e3f90 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149495 Aug 24 10:32:20.742169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 973/2084] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:20 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:20.875917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:20.876469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:20.882368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:20.882844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:20.883274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:20.883949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:20.884420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:20.884840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:20.948509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:20.957943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:20.959433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:20.960599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f8abff8b-0fa5-4472-a669-8a271bfe3887 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088621 Aug 24 10:32:20.961144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1112/2085] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:20 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:21.141188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:21.141760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:21.148900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:21.149384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:21.149823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:21.150515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:21.150969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:21.151443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:21.277988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:21.286104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e6c642ea-0a7a-47b3-95f4-a383a706340b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148918 Aug 24 10:32:21.286661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 974/2086] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:21 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:21.762534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:21.763624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:21.770147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:21.770635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:21.771072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:21.771768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:21.772253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:21.772677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:21.900029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:21.908749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d73d1c00-95f5-4166-9c88-f8bff0e570a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149604 Aug 24 10:32:21.909383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1113/2087] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:21 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:21.982134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:21.982738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:21.989750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:21.990233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:21.990688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:21.991375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:21.991823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:21.992272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:22.059567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:22.068439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:22.069929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:22.071236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0b4103bb-fbe2-44a7-880e-6ab40565731a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.093023 Aug 24 10:32:22.071795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 975/2088] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:21 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:22.307755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:22.308314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:22.314614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:22.315103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:22.315546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:22.316265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:22.316745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:22.317171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:22.443597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:22.451731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0d602bf8-d159-4df1-87a0-72d217f66e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147972 Aug 24 10:32:22.452326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1114/2089] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:22 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:22.930363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:22.930931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:22.937510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:22.937999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:22.938689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:22.940097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:22.940557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:22.940978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:23.077324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:23.086432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-70e8209f-bb57-44cf-a6ea-1b752da70d6f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.159962 Aug 24 10:32:23.086991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 976/2090] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:22 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:23.095105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:23.095709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:23.103204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:23.103704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:23.104173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:23.104937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:23.105341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:23.105810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:23.176417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:23.185373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:23.186797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:23.187945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1541b6d3-d6fc-4dbc-bd34-5b8ec1b73325 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.097105 Aug 24 10:32:23.188834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1115/2091] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:23 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:23.474339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:23.474879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:23.481086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:23.481574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:23.482041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:23.482733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:23.483187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:23.483649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:23.610125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:23.618208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9868a088-0170-42ea-abbe-3a8ba347d332 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148383 Aug 24 10:32:23.618764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 977/2092] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:23 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:24.108425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:24.108990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:24.116004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:24.116510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:24.116940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:24.117649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:24.118095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:24.118515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:24.210789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:24.211398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:24.218314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:24.218799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:24.219230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:24.219914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:24.220890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:24.221311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:24.247830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:24.256524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d8fae57b-9e3b-464d-859b-4fb3ebbeb729 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152345 Aug 24 10:32:24.257109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1116/2093] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:24 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:24.286211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:24.289605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:24.296279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:24.299109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:24.300467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b3e18dfc-3eac-482c-84bf-03aa27ed8f0b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.093544 Aug 24 10:32:24.300999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 978/2094] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:24 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:24.639253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:24.639868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:24.646595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:24.647088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:24.647531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:24.648266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:24.648724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:24.649157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:24.779864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:24.788277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-78c9816b-5ac5-4d9e-8b84-8922859ecc9b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152694 Aug 24 10:32:24.788853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1117/2095] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:24 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:25.277475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:25.278067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:25.280373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=70 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:25.285924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:25.286403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:25.286837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:25.287524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:25.287972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:25.288420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:25.322732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:25.323434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:25.329997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:25.330551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:25.331051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:25.331747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:25.332232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:25.332653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:25.398238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:25.407349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:25.409069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:25.410604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-630ad0e2-0cdc-42de-94bc-b369e8fdbfda tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091628 Aug 24 10:32:25.411285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1118/2096] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:25 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:25.420456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:25.429301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e87b4ca1-7af5-445e-ae61-58a3c49ec21e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155953 Aug 24 10:32:25.429856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 979/2097] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:25 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:25.809396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:25.810019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:25.816295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:25.816780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:25.817223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:25.817912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:25.818377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:25.818810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:25.944727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:25.952879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e68a1572-5ee7-4c1a-b896-e8d9805bf552 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147365 Aug 24 10:32:25.953450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1119/2098] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:25 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:26.433952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:26.434542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:26.441208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:26.441709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:26.442146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:26.442835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:26.443286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:26.443711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:26.450617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:26.451328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:26.458313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:26.458811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:26.459256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:26.459959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:26.460451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:26.460882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:26.506353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:26.515016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:26.516637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:26.517863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f8edf27e-c8b1-47b1-b34a-f916b7249755 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.087867 Aug 24 10:32:26.518496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 980/2099] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:26 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:26.588368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:26.591772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:26.598070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b1f3ee3f-09e5-4fb1-977f-191b4593f18d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151290 Aug 24 10:32:26.598636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1120/2100] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:26 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:26.974579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:26.975203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:26.981124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:26.981602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:26.982027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:26.982702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:26.983147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:26.983565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:27.114662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:27.122859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6c184dec-3747-4748-b551-4af25ab306da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152186 Aug 24 10:32:27.123426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 981/2101] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:26 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:27.540463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:27.541084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:27.543361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=77,nova_api:UPDATE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:27.549181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:27.549670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:27.550113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:27.550800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:27.551254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:27.551671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:27.616449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:27.618714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:27.619359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:27.625288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:27.625698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:27.626211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:27.627210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:27.627522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:27.627902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:27.628381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:27.628716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-80cc08b6-8029-48ca-8aed-e2e8edf69f68 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.092405 Aug 24 10:32:27.628825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:27.629613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1121/2102] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:27 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:27.761244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:27.769908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1310d05c-f597-473f-89c6-84b6fccd99de tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154987 Aug 24 10:32:27.770484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 982/2103] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:27 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:28.144769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:28.145302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:28.151260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:28.151746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:28.152223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:28.152911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:28.153362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:28.153784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:28.281633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:28.289716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6e4c62dd-2112-4540-ac24-d72ac1da6be5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149059 Aug 24 10:32:28.290270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1122/2104] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:28 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:28.650165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:28.650775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:28.657206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:28.657690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:28.658126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:28.658817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:28.659505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:28.660065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:28.720900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:28.729416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:28.731579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:28.732785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6c676eae-2338-47d5-be75-60bb75a1cbf2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.086527 Aug 24 10:32:28.733358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 983/2105] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:28 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:28.790980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:28.791590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:28.798781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:28.799280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:28.799713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:28.800424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:28.800877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:28.801296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:28.926719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:28.934889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2aa542a5-e90f-4d7b-a246-c8d002801588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147728 Aug 24 10:32:28.935701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1123/2106] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:28 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:29.310789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:29.311406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:29.318168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:29.318677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:29.319092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:29.319787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:29.320275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:29.320727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:29.449962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:29.458817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7d90f6dc-663c-450b-8fe3-519fc296f261 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151903 Aug 24 10:32:29.459386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 984/2107] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:29 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:29.755357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:29.755972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:29.761935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:29.762425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:29.762869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:29.763555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:29.764017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:29.764493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:29.829790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:29.838432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:29.839932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:29.841100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5050e28f-7a66-4d60-b51d-595a4812efee tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090286 Aug 24 10:32:29.841652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1124/2108] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:29.956804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:29.957415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:29.963878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:29.964392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:29.964826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:29.965510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:29.965964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:29.966385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:30.095731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:30.103798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9730218c-864d-4884-89d1-0839f95fc2d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151075 Aug 24 10:32:30.104385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 985/2109] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:29 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:30.480004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:30.480670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:30.487184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:30.487670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:30.488146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:30.488839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:30.489298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:30.489727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:30.614962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:30.623172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-500e9574-c424-46eb-ad56-a5cd1dbb52cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147034 Aug 24 10:32:30.623742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1125/2110] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:30 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:30.863381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:30.863988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:30.870000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:30.870481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:30.870917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:30.871605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:30.872063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:30.872496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:30.932713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:30.941658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:30.943167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:30.944343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-76c14a2c-b5f4-4687-8a6d-826c9b7e186c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.085497 Aug 24 10:32:30.944900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 986/2111] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:31.125139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:31.125684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:31.132131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:31.132627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:31.133060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:31.133741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:31.134196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:31.134623 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:31.260651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:31.269748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-18fd9e0e-5585-45b8-ac68-eec47985120f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148581 Aug 24 10:32:31.270339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1126/2112] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:31 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:31.645457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:31.645979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:31.652398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:31.652881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:31.653315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:31.654000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:31.654451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:31.654875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:31.785625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:31.793859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-78953ab0-15b5-4107-8c12-c3520d9e8211 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152980 Aug 24 10:32:31.794417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 987/2113] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:31 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:31.966899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:31.967504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:31.974148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:31.974624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:31.975056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:31.975726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:31.976216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:31.976641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:32.040632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:32.049335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:32.050855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:32.051993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4a13310f-6eca-4be5-9ec6-86c48f1240b8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089062 Aug 24 10:32:32.052593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1127/2114] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:31 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:32.290668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:32.291239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:32.297300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:32.297783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:32.298244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:32.298936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:32.299389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:32.299810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:32.428639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:32.436809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-219e0c47-8385-427e-befa-ea3eba0be15b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150107 Aug 24 10:32:32.437375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 988/2115] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:32 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:32.815638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:32.816291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:32.822653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:32.823139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:32.823578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:32.824294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:32.824753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:32.825182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:32.951942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:32.960231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7a3816b8-cb14-4e2b-9e0c-971f4f4f761b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149322 Aug 24 10:32:32.960799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1128/2116] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:32 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:33.073870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:33.074453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:33.080752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:33.081225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:33.081657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:33.082350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:33.082794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:33.083224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:33.145414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:33.153970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:33.155513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:33.156688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2d20e19e-f1ea-4b34-83a1-e33f3687c7b0 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.086852 Aug 24 10:32:33.157742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 989/2117] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:33.459090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:33.459698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:33.465671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:33.466149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:33.466581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:33.467257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:33.467704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:33.468157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:33.594254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:33.603010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bf1b9518-3994-4001-a413-8c5faaf2d996 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147768 Aug 24 10:32:33.603573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1129/2118] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:33 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:33.982271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:33.982935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:33.989503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:33.989979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:33.990406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:33.991082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:33.991526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:33.991940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:34.137594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:34.147051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f64dd7ba-90b5-48f8-969d-fbfc20e831d9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.168637 Aug 24 10:32:34.147625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 990/2119] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:33 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:34.178826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:34.179442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:34.185954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:34.186442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:34.186887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:34.187583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:34.188334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:34.188769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:34.253238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:34.262008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:34.263521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:34.264720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-90781a93-be56-45aa-98a3-662e6c91eccb tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089820 Aug 24 10:32:34.265280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1130/2120] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:34 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:34.624422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:34.624948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:34.631394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:34.631868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:34.632328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:34.633008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:34.633460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:34.633876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:34.764528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:34.773680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-02f436ce-3469-4ad6-89b1-309fa7ede9fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153249 Aug 24 10:32:34.774262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 991/2121] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:34 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:35.168692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:35.169312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:35.175737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:35.176245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:35.176687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:35.177369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:35.177824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:35.178250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:35.288147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:35.288751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:35.295156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:35.295531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:35.295959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:35.296678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:35.297124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:35.298366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:35.309741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:35.319804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-315d892e-9aec-48f6-875e-b96a0a3f82db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153886 Aug 24 10:32:35.319804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1131/2122] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:35 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:35.369103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:35.377786 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:35.379365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:35.380465 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e80161a3-d748-45f0-9b9f-1ba850dc4a04 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.097208 Aug 24 10:32:35.381022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 992/2123] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:35 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:35.794799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:35.795407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:35.801318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:35.801798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:35.802230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:35.802903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:35.803350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:35.803769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:35.927652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:35.935715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6ff0d96e-8cfc-4fe7-87ed-7e4db4a70246 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.144822 Aug 24 10:32:35.936305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1132/2124] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:35 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:36.339378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:36.340019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:36.346373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:36.346848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:36.347279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:36.347961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:36.348434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:36.348852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:36.403341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:36.404072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:36.411278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:36.411479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:36.411951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:36.412727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:36.413936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:36.413936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:36.479468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:36.480998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:36.488922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:36.489605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d897a95d-c772-47ea-bb54-6db5089b2461 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154052 Aug 24 10:32:36.490182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 993/2125] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:36 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:36.490416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:36.491813 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bf2653d6-160c-4d8e-934b-e836b7ab9a95 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.092902 Aug 24 10:32:36.492384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1133/2126] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:36 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:36.957273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:36.957903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:36.963794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:36.964312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:36.964750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:36.965441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:36.965918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:36.966347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:37.096119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:37.104494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d983391a-5972-4d2f-8fe9-674fa3617a68 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151112 Aug 24 10:32:37.105070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 994/2127] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:36 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:37.511158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:37.511782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:37.514461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:37.515077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:37.519257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:37.519740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:37.520198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:37.520889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:37.521339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:37.521761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:37.522088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:37.522576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:37.523012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:37.523706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:37.524202 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:37.524649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:37.542225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:37.590102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:37.599089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:37.601046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:37.601801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ced2b91d-1de0-473c-8d5b-a248686044d1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091290 Aug 24 10:32:37.602623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 995/2128] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:37.651970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:37.660314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fdf15b70-49f8-4f3a-bca0-2867216ecc92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153012 Aug 24 10:32:37.660868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1134/2129] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:37 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:38.125370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:38.126211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:38.133099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:38.133594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:38.134032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:38.134748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:38.135201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:38.135628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:38.270917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:38.279053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-836d76c8-a822-4ae7-a4be-05c2c54bbf61 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.157584 Aug 24 10:32:38.279616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 996/2130] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:38 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:38.624062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:38.624707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:38.631632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:38.632144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:38.632584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:38.633285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:38.633733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:38.634150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:38.680991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:38.681606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:38.688306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:38.688793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:38.689913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:38.690805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:38.691380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:38.691805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:38.698963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:38.707818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:38.709334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:38.710469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e50e7f9a-9784-42a4-b018-58eae6a03a35 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090457 Aug 24 10:32:38.711029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1135/2131] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:38 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:38.828094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:38.836480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bad4e3ab-30a9-465e-9592-765aa58022c3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.159275 Aug 24 10:32:38.837052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 997/2132] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:38 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:39.300475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:39.301078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:39.306955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:39.307431 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:39.307865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:39.308579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:39.309022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:39.309439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:39.438851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:39.447697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0416285c-0663-4b26-a3f1-8b4b7b56003f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151649 Aug 24 10:32:39.448303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1136/2133] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:39 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:39.732523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:39.733121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:39.739468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:39.739948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:39.740411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:39.741126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:39.741575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:39.741999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:39.806054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:39.815622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:39.817178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:39.818402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4d4acac3-c526-49d3-b412-11ecda7f272b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089796 Aug 24 10:32:39.819091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 998/2134] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:39 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:39.857358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:39.857986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:39.864410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:39.864894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:39.865337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:39.866025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:39.866476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:39.866903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:39.885176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:39.995657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:40.004433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ce106555-6a85-4740-a5f7-38a8f64aa241 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150952 Aug 24 10:32:40.004995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1137/2135] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:39 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:40.467793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:40.468515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:40.475169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:40.475651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:40.476109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:40.476802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:40.477263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:40.477688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:40.605359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:40.614283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-42ebfa06-89c7-4afe-9ded-c891b4d76dcc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150525 Aug 24 10:32:40.614842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 999/2136] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:40 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:40.840682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:40.841294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:40.847739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:40.848251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:40.848693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:40.849375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:40.849830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:40.850255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:40.913803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:40.922415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:40.923910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:40.925070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-385aa9bf-73be-433e-a192-db6efd586a8a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088292 Aug 24 10:32:40.925622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1138/2137] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:41.026068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:41.026607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:41.033598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:41.034173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:41.034504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:41.035192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:41.035641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:41.036078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:41.164354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:41.172459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b72a7068-bd57-4015-b34f-692d9c61f82e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150300 Aug 24 10:32:41.173019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1000/2138] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:41 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:41.634692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:41.635307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:41.641246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:41.641729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:41.642223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:41.642911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:41.643372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:41.643793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:41.769803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:41.777932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2fe15577-eaac-4f90-921d-f73663df745f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147113 Aug 24 10:32:41.778491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1139/2139] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:41 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:41.947457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:41.948086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:41.954031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:41.954513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:41.955692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:41.956414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:41.956873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:41.957300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:42.020838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:42.029455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:42.030964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:42.032135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f8cfaf1e-15e3-4f5d-a77a-caa1e8ba7158 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088604 Aug 24 10:32:42.032690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1001/2140] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:41 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:42.194317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:42.194934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:42.201409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:42.201892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:42.202333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:42.203017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:42.203470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:42.203923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:42.334451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:42.342669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-42dfec15-af36-4879-bce7-7dac324d5a9a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152248 Aug 24 10:32:42.343223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1140/2141] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:42 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:42.798585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:42.799192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:42.806396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:42.806876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:42.807306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:42.807987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:42.808463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:42.808881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:42.942343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:42.950840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-083366ce-5082-4f9d-baf1-efdaa8ec67a3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.156104 Aug 24 10:32:42.952007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1002/2142] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:42 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:43.056868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:43.057560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:43.064537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:43.065056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:43.065492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:43.066174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:43.066625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:43.067046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:43.130328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:43.138886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:43.140407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:43.141531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fe989697-95eb-462b-91b2-07f44bb92364 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090141 Aug 24 10:32:43.142642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1141/2143] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:43.363969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:43.364710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:43.372252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:43.372742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:43.373172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:43.373945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:43.374393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:43.374810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:43.506238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:43.514566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d12f30ac-57d4-4f19-ac76-1e50274863c6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154832 Aug 24 10:32:43.515132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1003/2144] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:43 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:43.972442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:43.973061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:43.979456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:43.979945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:43.980422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:43.981116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:43.981574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:43.982003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:44.113407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:44.121678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9ed4893b-272d-4c6e-86d1-8015a5bac74f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153834 Aug 24 10:32:44.122241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1142/2145] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:43 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:44.164700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:44.165313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:44.171679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:44.172198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:44.172638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:44.173429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:44.173890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:44.174367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:44.238233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:44.246866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:44.248674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:44.249817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aaae12d6-7276-4aab-be5d-95311775ebd9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089072 Aug 24 10:32:44.250381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1004/2146] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:44.536067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:44.536680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:44.542586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:44.543068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:44.543505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:44.544216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:44.544677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:44.545102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:44.671641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:44.680474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7c27679e-5d40-4d2c-995b-cddfe728082e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148184 Aug 24 10:32:44.681045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1143/2147] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:44 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:45.143927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:45.144491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:45.151010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:45.151489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:45.151922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:45.152635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:45.153087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:45.153508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:45.273705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:45.274341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:45.280506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:45.280999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:45.281237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:45.281437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:45.282136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:45.282593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:45.283185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:45.289787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cf170b28-cf3b-41e5-877e-1616941455a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150688 Aug 24 10:32:45.290437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1005/2148] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:45 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:45.347999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:45.356593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:45.358109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:45.359244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4b1da89b-4c43-4a6c-9179-b9231a74fbb2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090050 Aug 24 10:32:45.359800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1144/2149] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:45 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:45.702388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:45.703039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:45.709097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:45.709581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:45.710019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:45.710704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:45.711191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:45.711618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:45.838402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:45.847107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c8793f0e-12ef-4481-b3a3-a8bbb894989f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148597 Aug 24 10:32:45.847680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1006/2150] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:45 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:46.310845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:46.311471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:46.318441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:46.318928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:46.319368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:46.320123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:46.320578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:46.321007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:46.382886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:46.383480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:46.393995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:46.394546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:46.394952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:46.395705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:46.396173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:46.396694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:46.454425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:46.463066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:46.463224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-63840b93-db67-41c4-8661-b99a633e745e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.156233 Aug 24 10:32:46.463657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1145/2151] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:46 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:46.472663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:46.474268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:46.475411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f61435ae-ea29-44ca-ab36-eec5c695cb7a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.096550 Aug 24 10:32:46.475957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1007/2152] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:46 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:46.867581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:46.868318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:46.874441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:46.874932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:46.875370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:46.876729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:46.877197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:46.877621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:47.004169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:47.012371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8f9290a7-a7d6-4ca0-a36c-8332759994a7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148646 Aug 24 10:32:47.012931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1146/2153] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:46 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:47.483809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:47.484444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:47.491393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:47.493483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:47.493959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:47.494687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:47.495185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:47.495618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:47.498287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:47.498908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:47.505228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:47.505712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:47.506160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:47.506856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:47.507313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:47.507745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:47.572394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:47.582102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:47.583618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:47.584792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0de8fe22-5c8c-4c7d-97c3-de10b6df88b1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090441 Aug 24 10:32:47.585354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1147/2154] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:47.644114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:47.652300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2f824502-3d90-4bc5-8753-7e4a290cf01f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.172342 Aug 24 10:32:47.652975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1008/2155] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:47 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:48.033507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:48.034124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:48.040126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:48.040615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:48.041057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:48.041743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:48.042203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:48.042630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:48.170822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:48.179016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-48460793-e6c8-48f1-8be5-631a0584015e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149511 Aug 24 10:32:48.179582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1148/2156] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:48 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:48.607694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:48.608345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:48.615201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:48.615684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:48.616147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:48.616838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:48.617295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:48.617721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:48.674723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:48.675328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:48.680572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:48.681757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:48.682241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:48.682675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:48.683365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:48.683814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:48.684259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:48.689895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:48.691416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:48.692610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-02f51c50-e273-4d1e-863a-722b43aeee54 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088876 Aug 24 10:32:48.693170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1009/2157] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:48 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:48.816503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:48.824704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5524f257-3d7f-4a0c-9df7-072b6b50498c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154511 Aug 24 10:32:48.825264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1149/2158] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:48 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:49.200487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:49.201137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:49.207890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:49.208411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:49.208873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:49.209594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:49.210074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:49.210507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:49.342982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:49.351772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-775aef2e-c504-44bd-a7b6-09a8ed433505 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155220 Aug 24 10:32:49.352373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1010/2159] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:49 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:49.715156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:49.715755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:49.722785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:49.723269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:49.723706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:49.724426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:49.724875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:49.725295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:49.790693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:49.799347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:49.800851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:49.801979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2f2a9078-29d8-4be3-8a04-943d3fb768ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090802 Aug 24 10:32:49.802532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1150/2160] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:49.844385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:49.845604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:49.851540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:49.852023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:49.852492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:49.853186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:49.853641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:49.854065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:49.979009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:49.987532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-65ae8dd6-4001-4c2b-888a-58e53ed9a13b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.146988 Aug 24 10:32:49.988105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1011/2161] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:49 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:50.373101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:50.373660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:50.379720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:50.380238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:50.380677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:50.381365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:50.381836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:50.382242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:50.510920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:50.520148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-abc607bb-6529-48bd-b5bb-f78120d5fffa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151051 Aug 24 10:32:50.520708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1151/2162] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:50 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:50.824503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:50.825093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:50.831985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:50.832640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:50.833071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:50.833757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:50.834203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:50.834621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:50.838620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:50.896415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:50.906079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:50.908326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:50.909491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5ef86169-b20b-4aea-83ad-a18dc257a7aa tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088903 Aug 24 10:32:50.910044 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1012/2163] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:50 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:51.008412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:51.009023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:51.015342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:51.015824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:51.016298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:51.017552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:51.018009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:51.018438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:51.146290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:51.154339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-94d72f73-70e7-49ff-a59d-a12be170ae33 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149785 Aug 24 10:32:51.154895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1152/2164] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:51 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:51.541873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:51.542501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:51.548942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:51.549414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:51.549845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:51.550524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:51.550979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:51.551398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:51.680251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:51.688730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0d4d0010-e036-4dbf-9b90-6444b6486e94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150610 Aug 24 10:32:51.689134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1013/2165] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:51 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:51.931691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:51.932322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:51.938437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:51.939039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:51.939479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:51.940181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:51.940629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:51.941050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:52.004938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:52.014224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:52.015709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:52.017369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-966c834e-ca28-415e-8fa2-db93859b87c2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089620 Aug 24 10:32:52.017925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1153/2166] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:52.174631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:52.175250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:52.181793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:52.182216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:52.182652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:52.183343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:52.183794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:52.184274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:52.311648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:52.320294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aaf11beb-d789-4fc8-b9c0-2b99bd4a8292 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149501 Aug 24 10:32:52.320894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1014/2167] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:52 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:52.709760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:52.710381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:52.716403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:52.716871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:52.717313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:52.718000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:52.718459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:52.718894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:52.723458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:52.848830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:52.856964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-da1b4cde-562f-46e5-9696-6d00d403bce5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151152 Aug 24 10:32:52.857521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1154/2168] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:52 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:53.042402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:53.042957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:53.048950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:53.049432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:53.049861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:53.050597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:53.051059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:53.051527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:53.113138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:53.121950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:53.123526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:53.124684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f64c3b7f-30a4-421b-8824-c56d34f885a5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.086389 Aug 24 10:32:53.125231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1015/2169] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:53 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:53.342268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:53.342835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:53.349185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:53.349673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:53.350115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:53.350826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:53.351283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:53.351713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:53.480511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:53.489308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ada97d52-e152-49af-b1a8-eb245ea7f135 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150991 Aug 24 10:32:53.489880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1155/2170] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:53 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:53.877857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:53.878456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:53.884877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:53.885349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:53.885777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:53.886452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:53.886903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:53.887319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:54.022119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:54.030219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f2e0665b-e64b-47ba-b1ac-8fed80cf0240 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156216 Aug 24 10:32:54.030772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1016/2171] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:53 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:54.147487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:54.148127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:54.154015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:54.154496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:54.154931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:54.155607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:54.156079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:54.156500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:54.221265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:54.230025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:54.231505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:54.232668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-63d837f8-b9a8-4a25-a8f0-e2b1eeeffc65 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089094 Aug 24 10:32:54.233225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1156/2172] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:54 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:54.511115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:54.511728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:54.517843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:54.518403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:54.518839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:54.519530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:54.519981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:54.520435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:54.646017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:54.654150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-977a1b07-ee89-49d8-9c49-fa35a4fe32a6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147957 Aug 24 10:32:54.654708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1017/2173] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:54 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:55.050861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:55.051445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:55.058203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:55.058689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:55.059136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:55.059830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:55.060323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:55.060757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:55.188926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:55.197205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0cc7b106-78ed-42a8-b743-86bc5100fcfc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150328 Aug 24 10:32:55.197770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1157/2174] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:55 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:55.254859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:55.255462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:55.262226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:55.262781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:55.263214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:55.263876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:32:55.264037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:32:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:32:55.265746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:55.267253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:55.268170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:55.329618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:55.338399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:55.339889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:55.341056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-299e2546-3eb8-43fa-98dc-3d8840867911 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090135 Aug 24 10:32:55.341612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1018/2175] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:55 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:55.687490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:55.688145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:55.694117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:55.694602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:55.695041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:55.695726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:55.696212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:55.696640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:55.823990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:55.832864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-337e86d8-a1c4-4e36-9e3b-324b673091f7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149538 Aug 24 10:32:55.833456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1158/2176] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:55 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:56.217853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:56.218464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:56.220393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=80 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:56.225814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:56.226294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:56.226723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:56.227410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:56.227859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:56.228312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:56.357230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:56.364471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:56.365078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:56.366348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d4eaf29a-ef55-4603-9b74-9de7de47c467 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152387 Aug 24 10:32:56.366920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1019/2177] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:56 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:56.371416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:56.371894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:56.372363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:56.373043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:56.373493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:56.373925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:56.441629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:56.450375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:56.451883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:56.453057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-30b9172b-c96a-41f0-aab1-39a58b74ed9f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.092417 Aug 24 10:32:56.454312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1159/2178] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:56.854439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:56.855054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:56.861814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:56.862301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:56.862734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:56.863415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:56.863869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:56.864336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:56.991779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:57.000342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7e75b971-b0b6-49e0-9aa2-bea7fddc00a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149868 Aug 24 10:32:57.000914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1020/2179] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:56 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:57.387822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:57.388447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:57.395480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:57.395962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:57.396441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:57.397125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:57.397577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:57.398003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:57.476342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:57.476950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:57.483557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:57.484036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:57.484502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:57.485270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:57.485724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:57.486250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:57.526415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:57.535604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ad48daab-1310-4770-aa71-f52a28b85bf6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151610 Aug 24 10:32:57.536203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1160/2180] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:57 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:57.551313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:57.559997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:57.561537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:57.562940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a4a616a2-7bae-4588-b755-1b3d25c9859d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090531 Aug 24 10:32:57.563499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1021/2181] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:57 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:58.021773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:58.022326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:58.024160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=80 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:32:58.029280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:58.029761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:58.030201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:58.030886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:58.031341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:58.031765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:58.157283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:58.165413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cde8a507-6e34-4cb0-8a14-9fe73bb95e5c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147593 Aug 24 10:32:58.165996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1161/2182] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:58 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:58.556917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:58.557527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:58.564236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:58.564707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:58.565134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:58.565807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:58.566251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:58.566662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:58.585519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:58.586136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:58.592748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:58.593240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:58.593682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:58.594883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:58.595344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:58.595777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:58.660952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:58.669585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:58.671106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:58.672834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c0a4016c-f193-443c-9b78-f581590c5ff4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090646 Aug 24 10:32:58.673403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1162/2183] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:58 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:32:58.694563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:58.702695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e691e460-8065-443b-bb48-b90f61be0115 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149667 Aug 24 10:32:58.703245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1022/2184] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:58 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:32:59.187377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:59.188655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:59.194733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:59.195221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:59.195660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:59.196626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:59.197089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:59.197515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:59.323006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:59.331997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ace68c75-7e56-47b2-9b00-610794b84130 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149114 Aug 24 10:32:59.332590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1163/2185] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:59 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:32:59.696480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:59.697086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:59.703508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:59.703940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:59.704411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:59.705093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:59.705546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:59.705968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:59.724122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:32:59.724782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:32:59.731095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:59.731574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:59.732018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:59.732742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:32:59.733207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:32:59.733636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:32:59.769497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:59.776207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:59.777918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:59.779064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-be0d4d2c-82f7-41b7-b29f-e6253946f1ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.086578 Aug 24 10:32:59.779626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1023/2186] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:32:59 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:32:59.863271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:32:59.871380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-18a92abf-8ccc-4c1f-915b-77a48e23e729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151246 Aug 24 10:32:59.871937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1164/2187] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:32:59 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:33:00.352865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:00.353484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:00.360324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:00.360814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:00.361250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:00.361947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:00.362403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:00.362856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:00.493064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:00.501846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cb4cafc7-dccd-4327-8cb9-78986d5c2d5d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152885 Aug 24 10:33:00.503082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1024/2188] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:00 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:00.801918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:00.802533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:00.808947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:00.809434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:00.809873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:00.810561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:00.811018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:00.811447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:00.875595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:00.884498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:00.885993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:00.887149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ed5c09f5-4073-432d-bdc0-0c2895425edc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089211 Aug 24 10:33:00.887840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1165/2189] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:00 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:00.893002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:00.893615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:00.900011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:00.900524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:00.900958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:00.901648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:00.902096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:00.902523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:01.031087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:01.039804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a3beeb5d-7cd0-4fa8-a61e-875f7c3dfa4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150682 Aug 24 10:33:01.040411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1025/2190] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:00 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:01.524358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:01.524977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:01.530910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:01.531901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:01.532369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:01.533078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:01.533539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:01.533976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:01.661563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:01.669669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8ceb507c-5349-4332-8672-13f8e36b4e91 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149314 Aug 24 10:33:01.670236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1166/2191] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:01 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:01.909573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:01.910112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:01.916464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:01.916942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:01.917376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:01.918058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:01.918504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:01.918931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:01.979494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:01.988559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:01.990074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:01.991201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e6848dac-9239-443b-a785-3e0a10dbc09e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.085674 Aug 24 10:33:01.991751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1026/2192] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:02.060734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:02.061371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:02.068572 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:02.069046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:02.069478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:02.070150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:02.070601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:02.071018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:02.199836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:02.208407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-df26570f-1275-4a53-9aac-ec1d4d00e676 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151421 Aug 24 10:33:02.208973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1167/2193] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:02 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:02.692028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:02.692655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:02.698568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:02.699041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:02.699479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:02.700184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:02.700633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:02.701050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:02.837967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:02.846188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b7dfefd5-abdb-46f0-b24a-c918fa66348b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.159107 Aug 24 10:33:02.846756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1027/2194] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:02 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:03.014294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:03.014831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:03.020870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:03.021362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:03.021799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:03.022490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:03.022947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:03.023376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:03.103064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:03.112585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:03.114106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:03.115264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d9e94c63-1322-4c26-9203-3d5224ce135e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.105092 Aug 24 10:33:03.115821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1168/2195] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:03.230040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:03.230657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:03.238044 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:03.238530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:03.238969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:03.239661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:03.240144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:03.240575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:03.366006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:03.369447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:03.375405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-21bec889-f07f-4c4f-bc97-0d96a6b860e2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149276 Aug 24 10:33:03.375978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1028/2196] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:03 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:03.866513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:03.867120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:03.873287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:03.873815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:03.874255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:03.874951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:03.875473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:03.875861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:04.002372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:04.010564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-faefd4bf-a789-4fb9-b917-76532f2086b8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147896 Aug 24 10:33:04.011117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1169/2197] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:03 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:04.137211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:04.137731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:04.144163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:04.144639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:04.145068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:04.145750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:04.146195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:04.146612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:04.206905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:04.215357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:04.216874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:04.217997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ba7f1bc6-8ff0-464b-bc59-9469c924613a tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.084807 Aug 24 10:33:04.218650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1029/2198] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:04.396935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:04.397506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:04.412364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:04.412852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:04.413292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:04.413983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:04.414434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:04.414855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:04.543020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:04.546478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:04.552454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3a09c845-b3ec-44fd-a3a8-2ce1cd608cf3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.159413 Aug 24 10:33:04.553021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1170/2199] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:04 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:05.031754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:05.032321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:05.038727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:05.039212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:05.039654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:05.040383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:05.040840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:05.041268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:05.173979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:05.182311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b3214bc2-3935-491b-ad3d-f4c7342708b2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154489 Aug 24 10:33:05.182883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1030/2200] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:05 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:05.241153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:05.241761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:05.247599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:05.248115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:05.248563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:05.249236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:05.249684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:05.250104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:05.313511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:05.322065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:05.323553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:05.324718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a39ee734-2278-4308-b83a-c18f876fdefd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.087481 Aug 24 10:33:05.325264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1171/2201] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:05.574698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:05.575226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:05.581659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:05.582128 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:05.582575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:05.583252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:05.583698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:05.584150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:05.709996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:05.718111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ec5876e1-e0ec-43ea-9902-2c0ad9aecbfa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147334 Aug 24 10:33:05.719299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1031/2202] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:05 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:06.202685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:06.203423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:06.209449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:06.209937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:06.210384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:06.211074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:06.211534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:06.211969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:06.339647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:06.346721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:06.347323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:06.348894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-57a36155-90dc-4119-9618-9d75ffddd8e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150003 Aug 24 10:33:06.349473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1172/2203] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:06 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:06.353694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:06.354166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:06.354590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:06.355270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:06.355710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:06.356155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:06.423316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:06.434159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:06.435665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:06.437102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0c70eb53-73cd-4444-a120-903f69674811 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.094542 Aug 24 10:33:06.437792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1032/2204] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:06.740883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:06.741503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:06.747437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:06.747923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:06.748391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:06.749085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:06.749541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:06.749979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:06.876566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:06.884701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4e6ae1e3-24b1-4391-9b79-8fb3fb963106 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147717 Aug 24 10:33:06.885269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1173/2205] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:06 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:07.369024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:07.369607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:07.375914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:07.376424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:07.376854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:07.377529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:07.377973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:07.378387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:07.516027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:07.524612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1c217ceb-91e5-4c86-a155-d202f661d532 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.159425 Aug 24 10:33:07.525186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1033/2206] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:07 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:07.592980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:07.593654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:07.599506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:07.599992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:07.600463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:07.601147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:07.601602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:07.602030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:07.666495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:07.675320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:07.676856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:07.677989 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e615a733-c4ed-4ca1-8ce7-9a0df515d751 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.222088 Aug 24 10:33:07.678541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1174/2207] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:07 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:07.905628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:07.906240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:07.912258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:07.912735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:07.913166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:07.913844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:07.914287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:07.914700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:08.041442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:08.049977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-de99556d-d00d-451d-a637-204d2a083abd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148182 Aug 24 10:33:08.050532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1034/2208] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:07 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:08.545660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:08.546279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:08.553240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:08.553730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:08.554204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:08.554894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:08.555352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:08.555781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:08.683524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:08.692856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e520e1be-29e0-4e31-9d8f-3ab6ffd2c154 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151103 Aug 24 10:33:08.693412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1175/2209] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:08 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:08.701525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:08.702177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:08.708745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:08.709230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:08.709687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:08.710360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:08.710816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:08.711241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:08.776192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:08.785022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:08.786538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:08.787686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-01141f0f-bd43-4dea-977e-934e967abff2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090129 Aug 24 10:33:08.788273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1035/2210] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:08 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:09.072188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:09.072727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:09.078655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:09.079135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:09.079578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:09.080289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:09.080747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:09.081177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:09.207596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:09.215967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b0324515-0856-42e9-930e-7d3defce2d59 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147780 Aug 24 10:33:09.216566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1176/2211] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:09 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:09.714012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:09.714663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:09.721099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:09.721583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:09.722047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:09.722734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:09.723190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:09.723618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:09.810560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:09.811213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:09.818760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:09.819290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:09.819739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:09.820842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:09.820975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:09.821360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:09.855956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:09.864825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0ee3cc43-7226-4bc2-87f1-b3979d1b59be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154690 Aug 24 10:33:09.865582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1036/2212] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:09 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:09.890175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:09.899638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:09.901180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:09.902325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8b699d83-320d-4efe-8831-2a0783402eaf tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.095655 Aug 24 10:33:09.902887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1177/2213] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:09 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:10.237978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:10.238595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:10.244502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:10.244984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:10.245419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:10.246183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:10.246638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:10.247060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:10.374063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:10.382744 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8cb57f9c-3aa4-4397-adb9-426ee5bb214e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149196 Aug 24 10:33:10.383298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1037/2214] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:10 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:10.886293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:10.886913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:10.893417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:10.893901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:10.894344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:10.895030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:10.895489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:10.895920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:10.925130 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:10.925873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:10.932454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:10.933009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:10.933536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:10.934236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:10.934689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:10.935115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:11.001265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:11.010153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:11.011674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:11.012826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-59653f43-1039-459d-95d1-9a5dda1e5f16 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.092448 Aug 24 10:33:11.013377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1038/2215] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:10 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:11.024918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:11.033365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-210e90f4-1a50-4794-9f27-e768ab948c31 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150939 Aug 24 10:33:11.033928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1178/2216] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:10 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:11.403156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:11.403680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:11.410345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:11.410832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:11.411265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:11.411962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:11.412437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:11.412858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:11.545369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:11.558818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6ea191c3-b0dd-4052-81e4-338ea92c8408 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.159386 Aug 24 10:33:11.559470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1039/2217] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:11 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:12.034351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:12.034912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:12.041497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:12.041985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:12.042453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:12.043133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:12.043590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:12.044095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:12.054118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:12.054731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:12.061176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:12.061657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:12.062094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:12.063032 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:12.064342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:12.065089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:12.110072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:12.119504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:12.121023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:12.122161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-23a0e3ad-58a0-424c-9b3d-30e25e90e57d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091801 Aug 24 10:33:12.122893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1179/2218] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:12 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:12.195604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:12.203851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-461042f0-79c6-4817-be94-e8e1b191f863 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153430 Aug 24 10:33:12.204337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1040/2219] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:12 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:33:12.587980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:12.588627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:12.594573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:12.595292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:12.596336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:12.597027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:12.597482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:12.597907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:12.733814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:12.742444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e87744e0-a1ce-480c-9d08-26cd2d000f42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.158361 Aug 24 10:33:12.743007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1180/2220] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:12 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:13.144826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:13.145422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:13.151670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:13.152173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:13.152610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:13.153299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:13.153744 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:13.154179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:13.220693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:13.225047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:13.225812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:13.229493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:13.231043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:13.232159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0e4f3f55-8701-4e5d-97ea-01f1ab7d8939 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091324 Aug 24 10:33:13.232306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:13.232700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:13.232818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1041/2221] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:13 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:13.233173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:13.233875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:13.234336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:13.234766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:13.360478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:13.369170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f7d2b3d1-57af-44f8-b784-4c1be617c1e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147998 Aug 24 10:33:13.369735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1181/2222] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:13 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:13.764220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:13.764837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:13.771135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:13.771685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:13.772144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:13.772838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:13.773285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:13.773704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:13.900349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:13.908473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8170bdc8-8d49-4bc2-9232-ff631f2a4af4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148186 Aug 24 10:33:13.909063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1042/2223] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:13 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:14.257634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:14.258378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:14.265821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:14.266307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:14.266753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:14.267460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:14.267917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:14.268480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:14.334553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:14.343212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:14.344763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:14.345908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6532a898-a17e-45c5-94f2-e0a946166bd7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.094212 Aug 24 10:33:14.346467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1182/2224] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:14 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:14.390262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:14.390783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:14.397863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:14.398333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:14.398762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:14.399431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:14.399875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:14.400339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:14.526897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:14.534942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-698e6a51-e44e-4509-9f01-01500918196c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148714 Aug 24 10:33:14.535503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1043/2225] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:14 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:14.929907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:14.930513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:14.937216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:14.937705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:14.938145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:14.938841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:14.939550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:14.939981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:15.070152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:15.078359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-df2eff3f-470d-4db6-ade2-803f1bccca55 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152691 Aug 24 10:33:15.078919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1183/2226] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:14 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:15.368694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:15.369262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:15.375175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:15.375658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:15.376122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:15.376811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:15.377265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:15.377691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:15.442896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:15.451631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:15.453166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:15.454323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-63124528-15e2-43bf-a46d-fa2ede561a8e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089674 Aug 24 10:33:15.454878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1044/2227] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:15 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:15.556530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:15.557143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:15.563588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:15.564094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:15.564538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:15.565721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:15.566160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:15.567071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:15.693345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:15.701495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-43b78bee-e80c-446c-b630-8b4451f726e2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148857 Aug 24 10:33:15.702061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1184/2228] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:15 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:16.099438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:16.100037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:16.105982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:16.106455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:16.106883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:16.107556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:16.107999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:16.108457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:16.125515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:16.236199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:16.244437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-49adc7c3-645f-498c-862b-8b29c5d127d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149444 Aug 24 10:33:16.245024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1045/2229] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:16 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:16.477020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:16.477635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:16.483507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:16.483990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:16.484457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:16.485138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:16.485593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:16.486020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:16.546404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:16.555137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:16.556697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:16.557842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5645ce49-adc1-4292-b7a0-68a6187ce322 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.084980 Aug 24 10:33:16.558398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1185/2230] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:16 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:16.722763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:16.723434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:16.729926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:16.730406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:16.730841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:16.731524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:16.731977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:16.732434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:16.858214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:16.866859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1ed1aaea-4774-4cfa-b936-5a28f63c481f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147969 Aug 24 10:33:16.867422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1046/2231] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:16 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:17.264851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:17.265467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:17.272195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:17.272677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:17.273144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:17.273823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:17.274280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:17.274706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:17.405977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:17.414677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bf273d8b-e8c0-4c51-8e8c-5d4d13bd6060 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153746 Aug 24 10:33:17.415215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1186/2232] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:17 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:17.580781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:17.581312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:17.588621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:17.589109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:17.589549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:17.590359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:17.590821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:17.591247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:17.655379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:17.664949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:17.666490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:17.667628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6ad7d2ad-7408-48a1-ba42-27868752f0dd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090850 Aug 24 10:33:17.668214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1047/2233] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:17 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:17.888335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:17.888883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:17.894917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:17.895457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:17.895941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:17.897199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:17.897658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:17.898090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:17.915349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:18.024881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:18.033036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-52083b66-7915-44a4-85b7-cb0282053d63 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148659 Aug 24 10:33:18.033611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1187/2234] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:17 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:18.436398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:18.437009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:18.444387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:18.444871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:18.445307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:18.445992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:18.446451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:18.446875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:18.573260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:18.581529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8263d4cd-a250-42dd-9a58-3be5e7c052da tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149058 Aug 24 10:33:18.582099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1048/2235] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:18 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:18.689895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:18.690424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:18.697033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:18.697517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:18.697956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:18.698668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:18.699118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:18.699541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:18.768574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:18.777843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:18.779344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:18.780535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f20a18c1-5a3c-48e3-94e6-e965e7ff487c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.094618 Aug 24 10:33:18.781090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1188/2236] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:18 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:19.054048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:19.054578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:19.060557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:19.061041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:19.061479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:19.062163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:19.062614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:19.063037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:19.193417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:19.202223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d16fc164-e603-4fea-aec4-aebd031b5d04 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152157 Aug 24 10:33:19.202792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1049/2237] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:19 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:19.602944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:19.603542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:19.609491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:19.609971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:19.610401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:19.611077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:19.611524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:19.611944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:19.737948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:19.748529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-50b70dd6-1102-4e2f-8a9e-492f4cf1cf75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149701 Aug 24 10:33:19.749301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1189/2238] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:19 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:19.802812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:19.803415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:19.809300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:19.809770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:19.810221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:19.810888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:19.811332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:19.811745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:19.875089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:19.884163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:19.885668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:19.886794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e9489ab9-cfea-4ae9-8da8-4adb673eea10 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088041 Aug 24 10:33:19.887348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1050/2239] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:19 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:20.223644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:20.224214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:20.230683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:20.231197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:20.231765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:20.232573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:20.233036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:20.233468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:20.362697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:20.371636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a115e5c1-53c0-4714-a7f1-ea10bd76c58e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151911 Aug 24 10:33:20.372257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1190/2240] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:20 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:20.769794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:20.770413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:20.776956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:20.777443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:20.777882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:20.778568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:20.779023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:20.779447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:20.908513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:20.910170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:20.910809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:20.916915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:20.917125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-94977a12-813c-4d29-8ead-515defc809c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151238 Aug 24 10:33:20.917393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:20.917688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1051/2241] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:20 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:20.917858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:20.918505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:20.918958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:20.919379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:20.984133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:20.992873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:20.994378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:20.995499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-38780355-2dcc-4a14-90be-c46a91bf9240 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089754 Aug 24 10:33:20.996059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1191/2242] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:20 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:21.393056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:21.393662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:21.399778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:21.400283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:21.400709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:21.401379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:21.401827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:21.402244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:21.527055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:21.535116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5a107c6e-2633-4507-a695-d3b7ef69eb58 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.145989 Aug 24 10:33:21.535677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1052/2243] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:21 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:21.938045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:21.938662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:21.945190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:21.945668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:21.946106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:21.946792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:21.947248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:21.947670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:22.017749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:22.018382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:22.025397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:22.026056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:22.026224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:22.027067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:22.027343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:22.027755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:22.079533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:22.088787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b5827e02-f0ca-4f90-95ea-7a5b302dea3e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154514 Aug 24 10:33:22.089417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1192/2244] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:21 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:22.094239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:22.103711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:22.105258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:22.106424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-09c0bbb3-3bb9-4c6f-a5d4-895836ea7c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.092597 Aug 24 10:33:22.106980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1053/2245] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:22 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:22.556216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:22.556822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:22.562662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:22.563139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:22.563573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:22.564282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:22.564735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:22.565163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:22.691950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:22.701033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cfdf1340-4c8c-43fe-9cf6-9ec8aa6439c7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148808 Aug 24 10:33:22.701595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1193/2246] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:22 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:23.110477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:23.111129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:23.117624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:23.118601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:23.119036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:23.119724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:23.120216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:23.120644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:23.128362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:23.128975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:23.135870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:23.136392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:23.136830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:23.137517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:23.137971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:23.138394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:23.205321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:23.214600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:23.216145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:23.217311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fa195ad1-ca80-45ff-be0c-e47c5fcd1f7e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.092902 Aug 24 10:33:23.217862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1194/2247] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:23 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:23.253109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:23.261534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-273ec3c6-bdf7-47e4-b9bf-67df37a4689e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155032 Aug 24 10:33:23.262093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1054/2248] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:23 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:23.722088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:23.722710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:23.731101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:23.731949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:23.732751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:23.733585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:23.734196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:23.734842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:23.868995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:23.877381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ccf2ae36-6d01-4936-9d73-69980b793879 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.159173 Aug 24 10:33:23.877943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1195/2249] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:23 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:24.240340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:24.240982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:24.247987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:24.248497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:24.248933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:24.249623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:24.250078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:24.250501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:24.282432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:24.283191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:24.289815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:24.290306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:24.290754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:24.291458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:24.291918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:24.292374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:24.313869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:24.324698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:24.326689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:24.327622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bd9c8f39-5d09-4d6f-9531-3d15be17a0f8 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091264 Aug 24 10:33:24.328215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1055/2250] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:24 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:24.434896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:24.443264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e2e87919-3a5f-42b6-a7b6-41dcabe8c946 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.164765 Aug 24 10:33:24.443832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1196/2251] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:24 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:33:24.898766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:24.899394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:24.906049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:24.906527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:24.906958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:24.907636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:24.908112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:24.908533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:25.036562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:25.044665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2fdc289f-44bd-4ba3-a886-1090883f2466 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149835 Aug 24 10:33:25.045226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1056/2252] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:24 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:25.351821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:25.352386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:25.358744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:25.359258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:25.359696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:25.360412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:25.360867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:25.361294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:25.425607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:25.434879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:25.436411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:25.437546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-68b92838-7cab-461d-bb6d-bcf79c619e73 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089758 Aug 24 10:33:25.438099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1197/2253] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:25 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:25.465125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:25.465761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:25.472200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:25.472678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:25.473105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:25.473783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:25.474236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:25.474656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:25.603287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:25.611876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9e52731b-7836-460e-a453-4c0b9a7be97d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150653 Aug 24 10:33:25.612473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1057/2254] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:25 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:26.065980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:26.066598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:26.073749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:26.074187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:26.074625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:26.075316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:26.075765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:26.076222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:26.213322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:26.221393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-53610b57-85d8-4b02-97ad-8677cc6d6513 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.159271 Aug 24 10:33:26.221947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1198/2255] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:26 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:26.460166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:26.460768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:26.462662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=78 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:26.467899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:26.468555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:26.468825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:26.469505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:26.469952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:26.470524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:26.537149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:26.546215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:26.547695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:26.548860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b8b09524-a65d-4113-ba43-4ef9f45c3d08 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.092702 Aug 24 10:33:26.549409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1058/2256] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:26 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:26.633752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:26.634375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:26.641128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:26.641617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:26.642061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:26.642755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:26.643214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:26.643644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:26.771722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:26.780538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4658aa2d-342d-4b89-9b04-3c9c3bb5e765 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150532 Aug 24 10:33:26.781109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1199/2257] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:26 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:27.242800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:27.243414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:27.249891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:27.250377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:27.250819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:27.251505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:27.251961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:27.252413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:27.381707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:27.390523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2ac8c2ad-1f80-4398-8704-49a491a708a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151606 Aug 24 10:33:27.391084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1059/2258] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:27 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:27.571385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:27.571919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:27.578686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:27.579168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:27.579606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:27.580312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:27.580766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:27.581191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:27.646053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:27.654612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:27.656133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:27.657292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c04807bc-1378-4b60-bb60-0a9dbe5cfcb7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089922 Aug 24 10:33:27.657969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1200/2259] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:27 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:27.802211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:27.802824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:27.809846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:27.810327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:27.810765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:27.811455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:27.811910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:27.812362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:27.937147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:27.945246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ea9ced16-b64c-4892-87b3-4c3589c7f6f8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.146939 Aug 24 10:33:27.945803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1060/2260] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:27 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:28.411209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:28.411822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:28.413822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=80 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:28.419013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:28.419497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:28.419936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:28.420648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:28.421105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:28.421530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:28.546980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:28.555069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e12d20dd-a48c-48d9-907b-75a67c4990d7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147718 Aug 24 10:33:28.555639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1201/2261] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:28 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:28.679749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:28.680340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:28.686354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:28.686845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:28.687282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:28.687974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:28.688455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:28.688873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:28.749416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:28.757996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:28.759505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:28.760683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d63ed445-ad9e-41e5-b036-e36157238300 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.084917 Aug 24 10:33:28.761248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1061/2262] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:28 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:28.966447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:28.967062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:28.973446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:28.973922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:28.974355 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:28.975028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:28.975480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:28.975904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:29.102433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:29.110664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b8d6c3e4-5bc5-42a4-a620-5aadc9fa46ab tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148047 Aug 24 10:33:29.111232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1202/2263] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:28 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:29.576131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:29.576738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:29.583115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:29.583592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:29.584020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:29.584715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:29.585158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:29.585572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:29.589417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:29.714500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:29.723293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3ea10c3c-7e1d-41c8-ba8e-af79bc8e2369 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151414 Aug 24 10:33:29.723886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1062/2264] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:29 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:29.783313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:29.783963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:29.789968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:29.790490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:29.790937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:29.791634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:29.792119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:29.792552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:29.857817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:29.866474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:29.867990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:29.869155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9fc33016-31dd-4204-b779-6d4c9dd9741b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089815 Aug 24 10:33:29.869712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1203/2265] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:29 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:30.132293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:30.132834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:30.139764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:30.140282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:30.140722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:30.141416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:30.141866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:30.142291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:30.272689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:30.280788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c582a346-2e04-4297-a58e-5b54d90f88af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152569 Aug 24 10:33:30.281347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1063/2266] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:30 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:30.744857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:30.745470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:30.751780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:30.752289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:30.752726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:30.753404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:30.753854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:30.754274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:30.758156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:30.882533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:30.891144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:30.891699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7790b83a-ddcb-4a70-8dec-77754cd40084 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150326 Aug 24 10:33:30.891814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:30.891918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1204/2267] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:30 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:30.898104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:30.898582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:30.899014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:30.899694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:30.900169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:30.900591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:30.961588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:30.970890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:30.972442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:30.973564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e78e8b35-a4ba-43f8-9629-258fb6c0ee60 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.086854 Aug 24 10:33:30.974121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1064/2268] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:30 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:31.302015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:31.302645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:31.308591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:31.309082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:31.309526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:31.310220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:31.311242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:31.311678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:31.439129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:31.447233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f2a0442a-131f-4d61-85f0-477d33c35d81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149058 Aug 24 10:33:31.447787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1205/2269] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:31 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:31.912718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:31.913331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:31.919794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:31.920905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:31.921345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:31.922025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:31.922486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:31.922908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:31.995304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:31.995912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:32.002333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:32.002833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:32.003277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:32.003973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:32.004464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:32.004895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:32.054104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:32.062401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d121c080-6152-466e-ad95-67b9ac070a64 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153560 Aug 24 10:33:32.062974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1065/2270] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:31 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:32.070588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:32.079546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:32.081176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:32.082310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b531be1a-4fda-4506-b223-8067c593ff89 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090986 Aug 24 10:33:32.082870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1206/2271] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:31 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:32.469423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:32.470028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:32.475917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:32.476430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:32.476859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:32.477528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:32.477971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:32.478388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:32.607715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:32.616857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9e5c094b-d146-4401-889f-170c8ff8ab7a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151906 Aug 24 10:33:32.617443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1066/2272] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:32 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:33.083772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:33.084333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:33.090654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:33.091136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:33.091579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:33.092303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:33.092760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:33.093845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:33.105119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:33.105701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:33.112296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:33.112854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:33.113297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:33.113980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:33.114433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:33.114855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:33.181090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:33.190162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:33.191672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:33.192844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-714bb5ff-239e-432e-9a07-3268decc25ef tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091622 Aug 24 10:33:33.193873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1067/2273] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:33 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:33.227753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:33.236584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7dce8c04-bab1-40e8-b385-0db93b4e703f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.156404 Aug 24 10:33:33.238031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1207/2274] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:33 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:33.637612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:33.638260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:33.644752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:33.645232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:33.645669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:33.646350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:33.646806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:33.647228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:33.773459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:33.782306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0a33ba5b-bc87-49d0-a206-f9955a93dc81 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148532 Aug 24 10:33:33.786730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1068/2275] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:33 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:33:34.216141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:34.216763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:34.223135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:34.223620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:34.224084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:34.224776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:34.225237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:34.225669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:34.257881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:34.258660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:34.265491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:34.266584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:34.267019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:34.267709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:34.268208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:34.268630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:34.288519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:34.297453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:34.298994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:34.300185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3d7fce06-faee-4556-b61e-2f18a9924df1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088008 Aug 24 10:33:34.300747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1208/2276] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:34 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:34.394959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:34.403063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dd39ca95-a80c-40ef-a320-529b49cc01da tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149013 Aug 24 10:33:34.403629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1069/2277] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:34 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:33:34.803554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:34.804200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:34.810215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:34.810708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:34.811148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:34.811836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:34.812318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:34.812756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:34.947670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:34.956211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f763b2d5-24b5-4098-abb2-88c5fe0870eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156526 Aug 24 10:33:34.956764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1209/2278] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:34 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:35.322390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:35.323047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:35.330230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:35.330709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:35.331143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:35.331825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:35.332310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:35.332737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:35.396102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:35.404697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:35.406203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:35.407977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b77405e9-e730-4184-a9e0-1e24730b1ede tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089565 Aug 24 10:33:35.408554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1070/2279] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:35 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:35.422715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:35.423383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:35.429278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:35.429754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:35.430252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:35.430850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:35.431298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:35.431715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:35.563609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:35.571819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2a6a0b14-6f61-4539-bb69-96c69a1eabd0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153095 Aug 24 10:33:35.572484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1210/2280] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:35 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:35.977884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:35.978555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:35.984531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:35.985013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:35.986048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:35.986743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:35.987197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:35.987626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:36.115739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:36.124018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e282dea0-fcc0-4346-8cfa-ee492398bce4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149999 Aug 24 10:33:36.124608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1071/2281] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:35 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:36.429964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:36.430609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:36.437166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:36.437654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:36.438101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:36.438790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:36.439275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:36.439705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:36.501227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:36.509879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:36.511400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:36.512588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-420a7712-d0cd-4343-8f35-13d76fdde91b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.086624 Aug 24 10:33:36.513145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1211/2282] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:36 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:36.593006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:36.593618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:36.600795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:36.601279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:36.601715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:36.602397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:36.602849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:36.603269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:36.729551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:36.738359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8c5ae243-f06d-4f20-b524-21ef89dbb127 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149126 Aug 24 10:33:36.738935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1072/2283] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:36 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:37.148153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:37.148777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:37.154759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:37.155252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:37.155692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:37.156404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:37.156868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:37.157314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:37.323890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:37.333238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e062efb9-a8f8-4901-95c8-3e1b4691f734 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.188953 Aug 24 10:33:37.333837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1212/2284] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:37 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:37.535063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:37.535670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:37.541592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:37.542074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:37.542503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:37.543181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:37.543633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:37.544071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:37.619960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:37.629606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:37.631362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:37.633152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c1f63556-7ea7-423a-9854-3980c68dde22 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.102079 Aug 24 10:33:37.633978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1073/2285] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:37 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:37.759638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:37.760334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:37.766784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:37.767274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:37.767712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:37.768467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:37.768921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:37.769345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:37.962113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:37.971136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9a21bf4e-34df-417c-acf4-cbfe8bcc4730 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.215084 Aug 24 10:33:37.971748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1213/2286] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:37 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:38.354585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:38.355202 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:38.361473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:38.361961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:38.362400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:38.363097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:38.363551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:38.363983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:38.492592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:38.500667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-97d2e811-7fc7-4af6-87fe-d17df45dd042 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149920 Aug 24 10:33:38.501229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1074/2287] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:38 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:38.662009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:38.662546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:38.668585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:38.669073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:38.669515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:38.670205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:38.670665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:38.671099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:38.735423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:38.744442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:38.745959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:38.747106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1779f91a-aa12-4146-8636-d2b81af4540b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089157 Aug 24 10:33:38.747670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1214/2288] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:38 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:38.992747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:38.993366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:39.000124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:39.000612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:39.001046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:39.001736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:39.002187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:39.003282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:39.129506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:39.137628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-24c3dedc-d9fd-431d-9538-12f149141826 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148801 Aug 24 10:33:39.138190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1075/2289] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:38 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:39.521919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:39.522542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:39.528949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:39.529441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:39.529885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:39.530572 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:39.531064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:39.531493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:39.658762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:39.667530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-70a054f8-0800-4d42-ab8f-95b4a54d26c1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149567 Aug 24 10:33:39.668118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1215/2290] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:39 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:39.770721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:39.771569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:39.777279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:39.777750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:39.778184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:39.778858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:39.779305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:39.779722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:39.843950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:39.854198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:39.855719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:39.856912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-81efd2e9-36a1-45c0-9a08-4c6819438513 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090682 Aug 24 10:33:39.857470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1076/2291] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:39 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:40.158222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:40.159009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:40.165013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:40.165499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:40.165936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:40.166619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:40.167074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:40.167504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:40.295592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:40.303774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-913cb84b-90df-47c0-a6ff-e903e25c19ea tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149426 Aug 24 10:33:40.304382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1216/2292] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:40 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:40.687703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:40.688339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:40.694837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:40.695313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:40.695743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:40.696481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:40.696963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:40.697386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:40.826798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:40.835944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1c85cab0-ad86-4385-999e-649408740d6f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152047 Aug 24 10:33:40.836546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1077/2293] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:40 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:40.879814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:40.880443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:40.886774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:40.887246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:40.887675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:40.888382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:40.888832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:40.889251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:40.954617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:40.964406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:40.965910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:40.967050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d02d34ab-b29d-4d5f-8d02-a56823db55bc tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091618 Aug 24 10:33:40.967599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1217/2294] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:40 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:41.325378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:41.325906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:41.332613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:41.333087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:41.333512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:41.334205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:41.334630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:41.335046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:41.461702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:41.465174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:41.470549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d2fcb5ea-59bf-43d3-afa4-763b7f4df351 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149156 Aug 24 10:33:41.471111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1078/2295] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:41 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:41.857005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:41.857644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:41.863591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:41.864067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:41.864544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:41.865237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:41.865695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:41.866126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:41.990185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:41.990778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:41.999327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:41.999819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:42.000283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:42.000974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:42.001426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:42.001544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:42.001996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:42.010564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-805fd58d-96e9-4aa5-94b2-419b1566d9de tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.157386 Aug 24 10:33:42.011325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1218/2296] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:41 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:42.070691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:42.079283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:42.080828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:42.082211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d4099e56-6e05-4f53-9f52-0f3f9704413c tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.096402 Aug 24 10:33:42.082874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1079/2297] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:41 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:42.491532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:42.492173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:42.498090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:42.498576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:42.499016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:42.499705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:42.500187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:42.500614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:42.623290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:42.631872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8bdb76e8-5445-43ff-953c-5cc7a5ddb98d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.144312 Aug 24 10:33:42.632465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1219/2298] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:42 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:43.031797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:43.032419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:43.039878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:43.040395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:43.040828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:43.041520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:43.041963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:43.042441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:43.105035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:43.106218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:43.117175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:43.118049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:43.118837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:43.120087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:43.120921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:43.121687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:43.183873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:43.190993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:43.192330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d9de64f8-0d09-454f-9f31-aa5f666b87cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.164413 Aug 24 10:33:43.194667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1080/2299] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:43 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:43.194822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:43.203758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:43.205384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:43.206540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-79160209-3418-478c-ac27-637ca75d256f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.106111 Aug 24 10:33:43.207090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1220/2300] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:43 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:43.652819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:43.653436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:43.659308 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:43.659789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:43.660274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:43.661045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:43.662233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:43.662662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:43.792196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:43.800398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4b115c2c-6617-4390-8810-ff8af9d332a4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151389 Aug 24 10:33:43.800954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1081/2301] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:43 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:44.215575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:44.216223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:44.222490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:44.222975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:44.223522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:44.224854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:44.225315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:44.225742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:44.229939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:44.230660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:44.237067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:44.237544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:44.237977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:44.238660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:44.239294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:44.239540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:44.304766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:44.313453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:44.314952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:44.316113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-080c7491-d1ea-4b37-9ef2-1a5d486800af tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090054 Aug 24 10:33:44.316671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1082/2302] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:44 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:44.356291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:44.364566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d63509f4-fbc0-4b85-9396-dfbc3bce71dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153085 Aug 24 10:33:44.365168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1221/2303] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:44 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:44.821789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:44.822428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:44.828473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:44.828941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:44.829349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:44.830034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:44.830480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:44.830913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:44.959869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:44.968465 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dc69b4bf-1a80-42bb-98de-8d41e1f4c630 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151226 Aug 24 10:33:44.969021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1083/2304] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:44 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:45.339538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:45.340181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:45.346541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:45.347028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:45.347468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:45.348184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:45.348641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:45.349067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:45.385324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:45.385948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:45.392337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:45.392821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:45.393261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:45.393945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:45.394400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:45.394824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:45.416460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:45.425311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:45.426832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:45.428032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d4a5f22f-30c8-4f5d-895d-0da15eaaae1b tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.092493 Aug 24 10:33:45.429158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1222/2305] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:45 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:45.524549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:45.532641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e60b5008-dd10-4334-9fb9-a3f0a7304f41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151149 Aug 24 10:33:45.533208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1084/2306] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:45 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:33:45.990148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:45.991542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:45.997840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:45.998324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:45.998767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:45.999459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:45.999912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:46.000366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:46.129929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:46.138140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e733b000-7ffb-43b8-8669-e2dffae77000 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152898 Aug 24 10:33:46.138734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1223/2307] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:45 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:46.450805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:46.451391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:46.457985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:46.458463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:46.458993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:46.460555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:46.461006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:46.461426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:46.525746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:46.534398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:46.535905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:46.537184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9e5b9af7-3fb6-4c2a-a6d6-96ff27863a3f tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090276 Aug 24 10:33:46.537666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1085/2308] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:46 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:46.554110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:46.554751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:46.561095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:46.561577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:46.562017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:46.562701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:46.563165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:46.563590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:46.691268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:46.703003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-33cb695f-0c71-40e2-9ff1-7a8c377b3e4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152757 Aug 24 10:33:46.703572 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1224/2309] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:46 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:47.159134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:47.159656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:47.166409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:47.166881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:47.167314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:47.168011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:47.168481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:47.168892 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:47.294299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:47.302331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8d79e2cf-24c4-4259-a0df-4876b5e8f706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147150 Aug 24 10:33:47.302888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1086/2310] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:47 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:47.560898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:47.561511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:47.567899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:47.568410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:47.568941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:47.569627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:47.570084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:47.570511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:47.630818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:47.639386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:47.640927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:47.642055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3090ae9c-eba4-4a7e-8a5e-258c86024980 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.085116 Aug 24 10:33:47.642607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1225/2311] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:47 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:47.725551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:47.726768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:47.733213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:47.733690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:47.734119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:47.734798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:47.735247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:47.735666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:47.862248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:47.871781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a4259ef7-6d64-4003-8696-f5360b0b1252 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150194 Aug 24 10:33:47.872382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1087/2312] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:47 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:48.324074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:48.324694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:48.330547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:48.331031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:48.331464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:48.332159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:48.332624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:48.333030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:48.464846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:48.472976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dcb90754-510c-42e1-83d9-ac8e246753ed tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153240 Aug 24 10:33:48.473545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1226/2313] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:48 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:48.664177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:48.664705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:48.670911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:48.671392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:48.671826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:48.672537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:48.672990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:48.673416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:48.736684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:48.745731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:48.747243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:48.748412 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cd762bec-3ae5-4c2d-bd4b-2f404a8d1dff tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088226 Aug 24 10:33:48.748980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1088/2314] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:48 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:48.892783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:48.893409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:48.899926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:48.900440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:48.900911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:48.901600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:48.902058 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:48.902487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:49.030865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:49.039619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b2381a7f-9759-479d-bfe6-b8377eef8512 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150233 Aug 24 10:33:49.040208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1227/2315] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:48 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:49.494255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:49.494902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:49.501024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:49.501519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:49.501957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:49.502639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:49.503092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:49.503517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:49.629483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:49.637992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7bb561d3-e38c-47a5-baef-25a82a9c8067 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147736 Aug 24 10:33:49.638560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1089/2316] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:49 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:49.771139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:49.771543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:49.778445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:49.778923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:49.779356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:49.780032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:49.780504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:49.780924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:49.841442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:49.850549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:49.852073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:49.853198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-16fe4c9a-f708-49f5-849e-6bdc7106d3df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.086252 Aug 24 10:33:49.853759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1228/2317] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:49 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:50.060966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:50.061539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:50.067402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:50.067876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:50.068344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:50.069050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:50.069506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:50.069932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:50.198229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:50.206347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-00796099-1541-4da5-a683-225d8dd3ee5b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149279 Aug 24 10:33:50.206944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1090/2318] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:50 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:50.658548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:50.659175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:50.666234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:50.666722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:50.667170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:50.667860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:50.668352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:50.668783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:50.802055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:50.811211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f07f3be2-0d4b-46c7-9aee-e19781056f77 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.156604 Aug 24 10:33:50.811769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1229/2319] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:50 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:50.875583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:50.876224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:50.882772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:50.883244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:50.883671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:50.884375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:50.884852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:50.885296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:50.951261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:50.959809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:50.961332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:50.963232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5e946b35-ae67-4e36-b702-8956fdb7ee44 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091656 Aug 24 10:33:50.963780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1091/2320] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:50 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:51.228472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:51.229099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:51.235084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:51.235574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:51.236015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:51.236743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:51.237202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:51.237643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:51.363763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:51.371975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6fb42e11-efdc-48b4-93ab-98335e365a75 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147984 Aug 24 10:33:51.372567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1230/2321] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:51 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:33:51.832163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:51.832779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:51.839081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:51.839565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:51.839995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:51.840699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:51.841153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:51.841574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:51.967021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:51.975120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7da815e4-15b9-4aad-81d9-f285e3dfc1fb tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147027 Aug 24 10:33:51.975677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1092/2322] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:51 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:51.985413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:51.986031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:51.992425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:51.993069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:51.993524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:51.994718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:51.995179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:51.995613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:52.057117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:52.065722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:52.067230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:52.068403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a131f1da-2560-4468-9a19-7007a5e83b57 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.086944 Aug 24 10:33:52.069001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1231/2323] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:51 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:52.397701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:52.398227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:52.404836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:52.405317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:52.405753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:52.406453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:52.406901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:52.407323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:52.534017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:52.542125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-05adce3e-cadd-4d3f-aa06-b43e909e6141 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148418 Aug 24 10:33:52.542684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1093/2324] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:52 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:52.996472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:52.997094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:53.003486 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:53.003971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:53.004437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:53.005125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:53.005579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:53.006004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:53.092920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:53.093746 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:53.100213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:53.100706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:53.101144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:53.101852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:53.102313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:53.102743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:53.150359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:53.159868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-51081c67-db4f-4daa-818b-4b05c32acbc0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.167846 Aug 24 10:33:53.172684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1232/2325] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:52 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:53.177943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:53.186811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:53.188331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:53.189450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b804776f-9093-4204-924f-93d924a7a3bd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.100536 Aug 24 10:33:53.190011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1094/2326] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:53 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:53.563176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:53.563793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:53.569830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:53.570309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:53.570748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:53.571436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:53.572217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:53.572425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:53.696929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:53.705564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-15869f04-ccb4-428d-8e7b-ed9c0555165f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.146264 Aug 24 10:33:53.706128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1233/2327] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:53 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:54.181924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:54.183047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:54.189424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:54.189912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:54.190347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:54.191028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:54.191482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:54.191905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:54.210056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:54.210657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:54.211650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:54.218239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:54.218711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:54.219147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:54.219831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:54.220307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:54.220730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:54.286322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:54.295465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:54.297012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:54.298148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-eaa315d0-4b80-44a0-86be-53198495f4e2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091388 Aug 24 10:33:54.298724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1234/2328] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:54 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:33:54.322845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:54.331025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1eff817e-e156-4250-9d5e-e55630403dd4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153023 Aug 24 10:33:54.331577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1095/2329] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:54 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:54.727187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:54.727806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:54.734430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:54.734953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:54.735397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:54.736128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:54.736573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:54.737004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:54.865786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:54.873923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8ec738e7-5aa1-4951-8e67-5271a0c8936a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150700 Aug 24 10:33:54.874479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1235/2330] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:54 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:55.320895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:55.321494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:55.328534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:55.329018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:55.329451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:55.330131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:55.330590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:55.331014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:55.351749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:55.352389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:55.358725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:55.359208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:55.359647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:55.360357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:55.360815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:55.361252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:55.392921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:55.402433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:55.404002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:55.405175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b2ea2d5c-0afb-43e8-8b6e-7fa8e3ecc596 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088188 Aug 24 10:33:55.405741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1096/2331] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:55 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:55.489215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:55.497825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4953efcb-8649-4fea-a2a8-2c950bfc2d0d tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149973 Aug 24 10:33:55.498386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1236/2332] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:55 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:33:55.894747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:55.895362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:55.901362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:55.901846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:55.902289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:55.902968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:55.903424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:55.903845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:56.271589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:56.279820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dd526900-b312-477d-b51b-87490f640b1f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.388920 Aug 24 10:33:56.280459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1097/2333] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:55 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:56.426802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:56.427419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:56.434324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:56.434807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:56.435253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:56.435939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:56.436424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:56.436854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:56.455587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:56.503517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:56.512936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:56.514566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:56.515717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-262ce8df-a707-4cbc-bb80-9d26a9cdc774 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.092871 Aug 24 10:33:56.516304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1237/2334] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:56 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:56.519217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:56.519869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:56.522246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=80 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:56.527518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:56.528089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:56.528592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:56.529353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:56.529863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:56.530346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:56.654105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:56.662296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ce5dc261-6711-4c4c-9da2-fe0096191c09 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147240 Aug 24 10:33:56.662863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1098/2335] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:56 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:33:57.301320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:57.301931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:57.307850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:57.308373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:57.308812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:57.309496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:57.309949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:57.310379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:57.436713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:57.444739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c0621c0c-4acf-4afd-8934-2f3f5be950ea tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147247 Aug 24 10:33:57.445300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1238/2336] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:57 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:57.538476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:57.539118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:57.545656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:57.546136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:57.546575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:57.547260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:57.547713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:57.548165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:57.611472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:57.620305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:57.621875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:57.623009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c6d43c21-34bb-4918-8e00-c9c879a0fa6e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088516 Aug 24 10:33:57.623565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1099/2337] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:57 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:57.683234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:57.683861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:57.690300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:57.690780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:57.691215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:57.691891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:57.692380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:57.692804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:57.823301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:57.832003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c8a8f7c1-10b7-43bd-9bea-61690744caa9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152569 Aug 24 10:33:57.832601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1239/2338] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:57 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:58.466369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:58.466983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:58.472943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:58.473423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:58.473858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:58.474541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:58.474992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:58.475415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:58.604377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:58.612624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9fd8e706-8f68-488a-b2cd-21dc69b19568 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150442 Aug 24 10:33:58.613191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1100/2339] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:58 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:58.645655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:58.646264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:58.648556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=78 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:33:58.653731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:58.654203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:58.654644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:58.655322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:58.655809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:58.656223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:58.719759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:58.728405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:58.729950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:58.731092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9abc807e-36dd-44b5-ba56-638728d9d267 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.089295 Aug 24 10:33:58.731653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1240/2340] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:58 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:33:58.853272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:58.853916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:58.860475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:58.860956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:58.861396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:58.862074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:58.862530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:58.862955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:58.987500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:58.995556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f88e9b65-2ad6-40c6-ad47-f2ef305d6f1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.146175 Aug 24 10:33:58.996199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1101/2341] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:58 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:59.634278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:59.634904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:59.641565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:59.642035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:59.642476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:59.643162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:59.643615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:59.644056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:59.753174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:33:59.753787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:33:59.760161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:59.760652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:59.761091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:59.761776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:33:59.762229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:33:59.762653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:33:59.772646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:59.781610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c8a59342-6d08-4d07-90b1-effc314b224f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151185 Aug 24 10:33:59.782176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1241/2342] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:33:59 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:33:59.828456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:59.837076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:59.838586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:33:59.839715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-37c7618b-ce19-491e-bda1-b119a1c410f6 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090487 Aug 24 10:33:59.840306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1102/2343] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:33:59 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:34:00.017547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:00.018082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:00.024061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:00.024563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:00.025002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:00.025687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:00.026145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:00.026573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:00.152281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:00.160607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9e59eaa8-1079-441e-b83f-c7e51224287e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147683 Aug 24 10:34:00.161167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1242/2344] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:00 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:00.803323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:00.803937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:00.810376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:00.811391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:00.811833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:00.812542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:00.813016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:00.813425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:00.862743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:00.863356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:00.869741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:00.870230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:00.870671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:00.871361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:00.871828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:00.872305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:00.938516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:00.941624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:00.947547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:00.949112 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:00.949950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4f321e5e-d25f-4f91-908e-f83cb271c2e0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150545 Aug 24 10:34:00.950247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a997c832-1cc8-4add-902b-ae9cc3416004 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.091429 Aug 24 10:34:00.950522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1103/2345] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:00 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:00.950868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1243/2346] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:00 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:34:01.181322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:01.181941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:01.188600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:01.189082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:01.189517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:01.190201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:01.190773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:01.191194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:01.333721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:01.341961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-088b9329-be14-4919-8263-eff5ff57ea65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.164540 Aug 24 10:34:01.343638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1104/2347] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:01 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:01.972498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:01.973140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:01.973544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:01.973642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:01.979786 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:01.979936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:01.980297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:01.980413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:01.980726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:01.980842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:01.981404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:01.981554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:01.981848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:01.981969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:01.982593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:01.982723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:02.048038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:02.057162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:02.058658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:02.059806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c25cd3a8-8565-4d69-8a69-0c662e97cdbd tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.090793 Aug 24 10:34:02.060404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1105/2348] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:01 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:34:02.322772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:02.330977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d4e9b5bb-491b-4a4a-8bb1-6e953862c68a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.362430 Aug 24 10:34:02.331541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1244/2349] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:01 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:02.364547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:02.365196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:02.371069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:02.371548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:02.371984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:02.372697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:02.373145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:02.373589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:02.500094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:02.508235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-91097481-f1a6-4fb3-bfce-1db7190bdd4f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147474 Aug 24 10:34:02.508819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1106/2350] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:02 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:03.083150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:03.083725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:03.090099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:03.090586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:03.091028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:03.091745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:03.092390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:03.092661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:03.154310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:03.164779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:03.166581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:03.167438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b991f788-cc45-45f4-a120-a6ee522c71e5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088434 Aug 24 10:34:03.168020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1245/2351] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:03 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:03.352990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:03.353600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:03.360403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:03.360889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:03.361329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:03.362017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:03.362475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:03.362906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:03.492852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:03.501056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e87c31b9-84c8-4f91-aa0e-812315f67485 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152064 Aug 24 10:34:03.501618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1107/2352] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:03 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:03.528423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:03.529030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:03.535413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:03.535888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:03.536361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:03.537033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:03.537482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:03.537910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:03.664885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:03.673166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b9a00cdc-bc36-4079-99fd-ee9473a4164f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148511 Aug 24 10:34:03.673731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1246/2353] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:03 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:04.190386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:04.191015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:04.196958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:04.197439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:04.197873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:04.198551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:04.198998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:04.199418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:04.262669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:04.271923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:04.273430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:04.274593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-62ea77cc-e300-456b-bd66-a3f9f86b532e tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088145 Aug 24 10:34:04.275151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1108/2354] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:04 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:04.522919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:04.523569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:04.529910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:04.530394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:04.530833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:04.531522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:04.531977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:04.532429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:04.659986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:04.668143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bd24ed46-ac4a-4c29-967d-9b25c6afc040 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149205 Aug 24 10:34:04.668703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1247/2355] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:04 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:04.693545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:04.694152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:04.700548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:04.701026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:04.701458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:04.702137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:04.702588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:04.703008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:04.836500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:04.849136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6cfe5818-c9ba-470c-ac8d-178ae988ee1f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.159514 Aug 24 10:34:04.849726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1109/2356] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:04 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:05.297211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:05.297823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:05.303889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:05.304407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:05.304846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:05.305556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:05.306012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:05.306438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:05.366656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:05.375779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:05.377330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:05.378464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9dd0e90f-4912-4778-8290-b7ce2b6a27e9 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.085275 Aug 24 10:34:05.379013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1248/2357] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:05 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:05.689295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:05.689909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:05.695870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:05.696504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:05.696800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:05.697475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:05.697924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:05.698346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:05.825768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:05.833912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a25b59dd-5ca5-4490-8b51-b1cec7c7eb5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149113 Aug 24 10:34:05.834471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1110/2358] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:05 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:05.870384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:05.870998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:05.877661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:05.878143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:05.878578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:05.879265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:05.879721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:05.880178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:06.007919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:06.016097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8192d4fd-23d5-4c45-b986-a70344bfe830 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149492 Aug 24 10:34:06.016668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1249/2359] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:05 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:06.401650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:06.402262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:06.409343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:06.409842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:06.410284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:06.410975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:06.411433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:06.411904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:06.475206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:06.483964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:06.485545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:06.486682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b779313b-626e-4994-9af4-5646121449a3 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 200 len: 1445 microversion: 2.20 time: 0.088986 Aug 24 10:34:06.487237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1111/2360] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:06 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:34:06.508931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b8164791-5aa1-4f58-b46b-e0f40f8e42ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '.version_select at 0x7fa33d969700>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:06.530661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b8164791-5aa1-4f58-b46b-e0f40f8e42ac tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1318750226" status: 204 len: 0 microversion: 2.20 time: 0.025584 Aug 24 10:34:06.531221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1250/2361] 10.222.0.86 () {60 vars in 1262 bytes} [Wed Aug 24 10:34:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1318750226 => generated 0 bytes in 27 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 24 10:34:06.854949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:06.855569 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:06.862312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:06.862819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:06.863278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:06.863953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:06.864426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:06.864841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:06.868805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:07.036958 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:07.037683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:07.044004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:07.044525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:07.044961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:07.045650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:07.046100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:07.046520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:07.066965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:07.076010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e79474b7-3091-44cb-a847-a33651b569dc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.224953 Aug 24 10:34:07.076630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1112/2362] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:06 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:07.193685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:07.203411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-366028f9-6075-46f3-8cdd-a3870395cf6c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.170324 Aug 24 10:34:07.204031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1251/2363] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:07 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:07.373441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1da23d22-a8ae-4971-89f8-ce5a683a6c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Action: 'create', calling method: .version_select at 0x7fa33d77e430>, body: {"keypair": {"name": "tempest-keypair-379029465", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB90jFwXRxaUi5eEJI/vRL4r6vjZShMO9UAPTlrImL6ckhlaltrAr9geLfwbXAcTZCqCZMMIv4z5BXJFv//Q58dbYPZxW/5MkL2YbDIDbRWRkIXBCoTWJPcPmKkreNMvsA=="}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:34:07.388877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-1da23d22-a8ae-4971-89f8-ce5a683a6c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Getting quotas for project 7c1ad7a7198d4e78b9d03f00ce65e091. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:34:07.393279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-1da23d22-a8ae-4971-89f8-ce5a683a6c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Getting quotas for user df9b402589fa4020bc241a4b9c06dd91 and project 7c1ad7a7198d4e78b9d03f00ce65e091. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:34:07.418591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1da23d22-a8ae-4971-89f8-ce5a683a6c37 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.049007 Aug 24 10:34:07.419188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1113/2364] 10.222.0.86 () {62 vars in 1200 bytes} [Wed Aug 24 10:34:07 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 50 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 24 10:34:08.097223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:08.097821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:08.104252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:08.104739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:08.105175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:08.105862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:08.106315 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:08.106738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:08.225869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:08.226591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:08.241463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:08.241463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:08.242820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:08.243648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:08.244287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:08.244723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:08.329046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:08.339136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2e74ea3d-a27e-45f8-8cd2-6324ca8ab701 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.245957 Aug 24 10:34:08.340266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1252/2365] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:08 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:08.411297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:08.423062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-16085a8f-810a-450a-a02a-39a5ff17963b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.201224 Aug 24 10:34:08.424201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1114/2366] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:08 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:08.583365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:08.585205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:08.591780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:08.593188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:08.593847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:08.594660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:08.595327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:08.596066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:08.621951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:34:08.671054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:08.708887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-353262dd-f804-4e47-8aec-7072cfb249bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 204 len: 0 microversion: 2.1 time: 0.129007 Aug 24 10:34:08.712661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1253/2367] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:34:08 2022] DELETE /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:34:08.719293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:08.720198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:08.727436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:08.727925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:08.728397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:08.729103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:08.729554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:08.729975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:08.792419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-898130417", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "26b92eb1-90dd-4030-8ddb-d65fcb93c009"}], "security_groups": [{"name": "tempest-securitygroup--1256671632"}], "key_name": "tempest-keypair-379029465", "user_data": "}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:34:08.882593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:08.890808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-20bdb27d-f38a-43a9-8b42-78892b08d35c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.177007 Aug 24 10:34:08.891384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1115/2368] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:08.902390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] validate_networks() for [('26b92eb1-90dd-4030-8ddb-d65fcb93c009', None, None, None, None, None)] {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:34:09.362627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:09.363205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:09.369610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:09.370089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:09.370586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:09.371278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:09.371731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:09.372185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:09.380441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:34:09.380923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:34:09.381335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:34:09.381754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:34:09.382168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:34:09.390928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:09.391414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:09.391853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:09.392622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:09.393083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:09.393511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:09.410454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Getting quotas for project 7c1ad7a7198d4e78b9d03f00ce65e091. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:34:09.414507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Getting quotas for user df9b402589fa4020bc241a4b9c06dd91 and project 7c1ad7a7198d4e78b9d03f00ce65e091. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:34:09.426325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:34:09.428802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:34:09.480335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-34a4d230-47b6-4e07-9945-17c78f9e6946 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.692898 Aug 24 10:34:09.487370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1254/2369] 10.222.0.86 () {62 vars in 1188 bytes} [Wed Aug 24 10:34:08 2022] POST /compute/v2.1/servers => generated 402 bytes in 695 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Aug 24 10:34:09.487370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:09.487370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:09.493913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:09.494408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:09.494848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:09.495545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:09.496004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:09.496459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:09.513291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:09.526274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-61d56407-7d38-4896-bdbf-92b1e54798c9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.166214 Aug 24 10:34:09.541350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1116/2370] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:09 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:09.550087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e7342b08-5641-46be-b97f-128207ceb9a2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:09.550700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e7342b08-5641-46be-b97f-128207ceb9a2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:09.606702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e7342b08-5641-46be-b97f-128207ceb9a2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:09.616034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e7342b08-5641-46be-b97f-128207ceb9a2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:09.617547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e7342b08-5641-46be-b97f-128207ceb9a2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:09.618692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e7342b08-5641-46be-b97f-128207ceb9a2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1280 microversion: 2.20 time: 0.072995 Aug 24 10:34:09.619439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1117/2371] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:09 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1280 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:09.638652 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:09.646890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2eb01864-e62d-4c55-90fb-cd9a1f1b92e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.164945 Aug 24 10:34:09.647459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1255/2372] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:09 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:09.912648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:09.913303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:09.919779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:09.920358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:09.920789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:09.921480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:09.921927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:09.922347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:10.062768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:10.071023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-04aecf47-27c1-45dc-a904-48ac3c06e267 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.162729 Aug 24 10:34:10.078902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1118/2373] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:09 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:10.547522 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:10.548231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:10.554604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:10.555087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:10.555531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:10.556281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:10.556784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:10.557217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:10.641205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:10.641895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:10.649044 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:10.649661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:10.650100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:10.650796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:10.651279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:10.651706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:10.689675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:10.698621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9c6eb09b-df60-4601-a2ba-2b6fdbe5ef05 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154957 Aug 24 10:34:10.699198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1256/2374] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:10 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:10.703582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:10.704320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:10.711155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:10.711862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:10.713043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:10.713931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:10.714387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:10.714855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:10.715629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:10.726583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:10.728273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:10.729585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f4e10ec6-7173-49c0-bea0-b5caef11a549 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1332 microversion: 2.20 time: 0.092256 Aug 24 10:34:10.730354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1119/2375] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:10 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1332 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:34:10.925241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:10.936082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d554c541-57bc-46fb-a675-4aa2422c0296 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.235804 Aug 24 10:34:10.936950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1257/2376] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:10 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:11.092686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:11.093218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:11.099608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:11.100110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:11.100576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:11.101258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:11.101710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:11.102133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:11.236837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:11.246544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dcbd23d5-7c9f-4c7a-b190-50d61875232c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.157790 Aug 24 10:34:11.247216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1120/2377] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:11 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:11.720715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:11.721431 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:11.728171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:11.728931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:11.729372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:11.730272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:11.730735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:11.731164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:11.754010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:11.755218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:11.762251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:11.763208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:11.765082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:11.765978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:11.766702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:11.767317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:11.910541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:11.919374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:11.921027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:11.922240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-65ff4217-0f39-499b-9ab0-8fa0afcd67df tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1406 microversion: 2.20 time: 0.173939 Aug 24 10:34:11.922874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1121/2378] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:11 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1406 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:11.957864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:11.958568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:11.964598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:11.965082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:11.965521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:11.966212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:11.966663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:11.967087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:11.975155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:11.983923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-93915a26-0fc5-4b8a-95c8-211a8d81cec1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.266949 Aug 24 10:34:11.984723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1258/2379] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:11 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:12.105540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:12.114356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b027c654-13a2-46d6-bfc0-b640fedb92e8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.160357 Aug 24 10:34:12.115111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1122/2380] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:11 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:12.268707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:12.270136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:12.275968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:12.276495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:12.276942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:12.277637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:12.278098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:12.278552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:12.382417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-ad0df09e-70dd-4d54-85be-6bd8f51678a1 req-41887d04-3ccf-4683-9c8e-07985af1c9ea service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9b34f307-a219-4d5f-aa5d-0e3eb79ea936", "tag": "b7060ba2-b694-4997-b4c4-21985557243b"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:34:12.393184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-ad0df09e-70dd-4d54-85be-6bd8f51678a1 req-41887d04-3ccf-4683-9c8e-07985af1c9ea service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:12.393787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-ad0df09e-70dd-4d54-85be-6bd8f51678a1 req-41887d04-3ccf-4683-9c8e-07985af1c9ea service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:12.394944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-ad0df09e-70dd-4d54-85be-6bd8f51678a1 req-41887d04-3ccf-4683-9c8e-07985af1c9ea service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:12.406586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-ad0df09e-70dd-4d54-85be-6bd8f51678a1 req-41887d04-3ccf-4683-9c8e-07985af1c9ea service nova] Creating event network-changed:b7060ba2-b694-4997-b4c4-21985557243b for instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 on hv-cinder-85427 Aug 24 10:34:12.411887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-ad0df09e-70dd-4d54-85be-6bd8f51678a1 req-41887d04-3ccf-4683-9c8e-07985af1c9ea service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034344 Aug 24 10:34:12.412592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1123/2381] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:34:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:34:12.524662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:12.532876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2775ef23-a7af-42cd-8a45-6596f09a00ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.268171 Aug 24 10:34:12.533450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1259/2382] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:12.946811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:12.947575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:12.953807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:12.954290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:12.954736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:12.955422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:12.955924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:12.956388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:13.007071 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:13.007692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:13.013869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:13.014363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:13.014804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:13.015506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:13.015965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:13.016425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:13.095614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:13.104378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:13.105926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:13.107035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8d1476bc-9235-4c85-ba09-981c83ff8207 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1406 microversion: 2.20 time: 0.164976 Aug 24 10:34:13.107587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1124/2383] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:12 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1406 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:13.136368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:13.137194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:13.143670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:13.144228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:13.144706 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:13.145401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:13.145853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:13.146276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:13.152214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:13.160646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e1b673d6-0484-4956-8617-2ac192ef458e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.158323 Aug 24 10:34:13.161211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1260/2384] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:13 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:13.277081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:13.285273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fdfadccc-9540-4ae6-9f48-db38c1242646 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152874 Aug 24 10:34:13.285894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1125/2385] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:13 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:13.554775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:13.555348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:13.563843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:13.564370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:13.564810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:13.565496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:13.565953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:13.566378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:13.695602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:13.703752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0f5ac359-0d77-4943-bd3b-a821cbb3b0c6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152951 Aug 24 10:34:13.704350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1261/2386] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:13 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:14.130187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:14.130804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:14.137623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:14.138115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:14.138553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:14.139257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:14.139712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:14.140191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:14.181778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:14.182378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:14.188802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:14.189281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:14.189711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:14.190393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:14.191035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:14.191629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:14.278239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:14.286523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:14.288076 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:14.289209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9ddb2140-98dc-4d8a-8731-209afc3f61d4 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1406 microversion: 2.20 time: 0.163072 Aug 24 10:34:14.289765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1126/2387] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:14 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1406 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:14.309226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:14.309834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:14.315721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:14.316369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:14.317235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:14.317866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:14.318319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:14.318740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:14.406593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:14.415654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f852fb23-1654-4464-9c39-c5defda559a3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.237575 Aug 24 10:34:14.416672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1262/2388] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:14 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:14.454030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:14.462845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-47ada9b2-a366-4752-aae7-776eccfd7999 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.157402 Aug 24 10:34:14.463404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1127/2389] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:14 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:14.725630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:14.726152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:14.732227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:14.732708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:14.733145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:14.733821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:14.734271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:14.734693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:14.867346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:14.875631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-134d6868-b36e-44ea-9391-2677e6b8a212 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154287 Aug 24 10:34:14.876229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1263/2390] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:14 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:15.312474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:15.313068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:15.319462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:15.319938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:15.320398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:15.321090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:15.321526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:15.321941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:15.437286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:15.438020 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:15.444521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:15.445100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:15.445615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:15.446384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:15.446916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:15.447441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:15.453424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:15.462432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:15.463939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:15.465056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ab044cb0-8f2a-4503-adc9-d633e317db29 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1406 microversion: 2.20 time: 0.156803 Aug 24 10:34:15.465700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1128/2391] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:15 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1406 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:15.483576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:15.484336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:15.490318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:15.490786 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:15.491206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:15.491879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:15.492344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:15.492759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:15.580394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:15.589310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c2a56788-28f9-4272-a7ac-15f86c4b1919 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155874 Aug 24 10:34:15.589886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1264/2392] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:15 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:15.623176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:15.631274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a8b5ff45-8504-48a3-b875-b58571bae3f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151454 Aug 24 10:34:15.631828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1129/2393] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:15 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:15.898107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:15.898627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:15.904637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:15.905134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:15.905564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:15.906252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:15.906712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:15.907138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:16.032571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:16.040736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-201e028a-be30-44bd-a486-be4e06a35b75 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.147309 Aug 24 10:34:16.041510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1265/2394] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:15 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:16.488630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:16.489267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:16.495876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:16.496395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:16.496833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:16.497531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:16.497990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:16.498419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:16.611678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:16.612328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:16.618902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:16.619647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:16.620093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:16.620794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:16.621279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:16.621672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:16.634244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:16.643139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:16.645004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:16.647023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fb7eb0ac-f02c-4dee-97d3-ebbb2db0d290 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1406 microversion: 2.20 time: 0.162474 Aug 24 10:34:16.647768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1130/2395] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:16 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1406 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:16.652883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:16.653830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:16.670913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:16.671686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:16.672154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:16.672859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:16.673312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:16.673743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:16.692228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:16.814166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:16.823951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f94a8f95-ba36-4617-b4d3-74e01ee70bb2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.174902 Aug 24 10:34:16.824602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1131/2396] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:16 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:17.008915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:17.017209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-be513a4e-de76-432c-b926-4ab44743a5e3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.409482 Aug 24 10:34:17.017771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1266/2397] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:16 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:17.063011 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:17.063625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:17.070450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:17.070928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:17.071364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:17.072068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:17.072514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:17.072936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:17.212675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:17.220701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1fd1bf9e-7e07-4eff-8ccc-6b2ac53eac19 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.161733 Aug 24 10:34:17.221276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1132/2398] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:17 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:17.671550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:17.672179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:17.678541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:17.679016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:17.679455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:17.680170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:17.680619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:17.681043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:17.807834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:17.815943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:17.817459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:17.818550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d0c5f5dc-a152-443d-bbf8-62fdc8b2fdb1 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1406 microversion: 2.20 time: 0.151006 Aug 24 10:34:17.819096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1267/2399] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:17 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1406 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:17.843586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:17.844208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:17.851750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:17.852273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:17.852714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:17.853451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:17.853901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:17.854321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:17.984710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:17.992906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-166a1584-320c-428b-a6ef-e27fa85de6d6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153147 Aug 24 10:34:17.993477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1133/2400] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:17 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:18.038336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:18.038864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:18.044795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:18.045273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:18.045706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:18.046378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:18.046826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:18.047246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:18.182528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:18.191739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d954537c-c67e-449a-b682-ef8508d15f00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.157213 Aug 24 10:34:18.192489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1268/2401] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:18 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:18.242603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:18.243261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:18.249894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:18.250390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:18.250830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:18.251667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:18.253335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:18.253335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:18.320789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-dcc622cb-2641-4c1c-b6ac-106df007962d req-17401449-c87c-42ae-a991-563e661d1407 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9b34f307-a219-4d5f-aa5d-0e3eb79ea936", "name": "network-vif-plugged", "status": "completed", "tag": "b7060ba2-b694-4997-b4c4-21985557243b"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:34:18.330158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-dcc622cb-2641-4c1c-b6ac-106df007962d req-17401449-c87c-42ae-a991-563e661d1407 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:18.330635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-dcc622cb-2641-4c1c-b6ac-106df007962d req-17401449-c87c-42ae-a991-563e661d1407 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:18.331068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-dcc622cb-2641-4c1c-b6ac-106df007962d req-17401449-c87c-42ae-a991-563e661d1407 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:18.342144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-dcc622cb-2641-4c1c-b6ac-106df007962d req-17401449-c87c-42ae-a991-563e661d1407 service nova] Creating event network-vif-plugged:b7060ba2-b694-4997-b4c4-21985557243b for instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 on hv-cinder-85427 Aug 24 10:34:18.347261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-dcc622cb-2641-4c1c-b6ac-106df007962d req-17401449-c87c-42ae-a991-563e661d1407 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.113885 Aug 24 10:34:18.347969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1134/2402] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:34:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:34:18.392548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:18.400758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bdd670da-baad-41f3-b1b9-31cc9796c80e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.162216 Aug 24 10:34:18.401313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1269/2403] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:18 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:18.840480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:18.841101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:18.847209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:18.847812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:18.848141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:18.848827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:18.849674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:18.849674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:18.977769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:18.985806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:18.987311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:18.988468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c554c58e-7099-4d65-a50d-93ba49f0f890 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1406 microversion: 2.20 time: 0.152447 Aug 24 10:34:18.989031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1135/2404] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:18 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1406 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:19.014260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:19.014877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:19.021378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:19.022344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:19.022792 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:19.023477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:19.023932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:19.024381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:19.150539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:19.154048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:19.159533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cc33446c-c711-40af-984b-6b999b7a6c39 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149176 Aug 24 10:34:19.160119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1270/2405] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:19 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:19.214056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:19.214773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:19.220751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:19.221233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:19.221668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:19.222351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:19.222805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:19.223227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:19.353789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:19.361844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-46a90abb-bd5d-4804-810d-84898e01cd7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152175 Aug 24 10:34:19.362411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1136/2406] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:19 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:19.421766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:19.422375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:19.424657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:19.425282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=6 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:19.431435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:19.431913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:19.432373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:19.433055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:19.433507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:19.433930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:19.574115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:19.582360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1413e9cd-4a36-4f8e-ae03-cc0b7a726c2d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.164392 Aug 24 10:34:19.583001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1271/2407] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:19 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:20.012679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:20.013289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:20.015284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=6 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:20.020887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:20.021558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:20.021999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:20.022707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:20.023154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:20.023575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:20.150764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:20.158791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:20.160316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:20.161401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ff6e935b-e65b-4923-9c63-0d72b5b5c202 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1406 microversion: 2.20 time: 0.152713 Aug 24 10:34:20.161957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1137/2408] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:20 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1406 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:20.181917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:20.182554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:20.188875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:20.189359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:20.189821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:20.190509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:20.190962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:20.191381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:20.327653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:20.335996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d4ce4dc4-2090-48a8-b8b9-480700571440 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.158596 Aug 24 10:34:20.339788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1272/2409] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:20 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:20.383497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:20.384135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:20.390599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:20.391083 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:20.391522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:20.392260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:20.392713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:20.393137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:20.524718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:20.532883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7d8ddca2-2c1e-4298-97f6-0b855dde21c7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153261 Aug 24 10:34:20.533436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1138/2410] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:20 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:20.604464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:20.605074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:20.610936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:20.611411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:20.611848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:20.612556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:20.613007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:20.613428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:20.739734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:20.747925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-10677134-44f4-46f6-9823-02b66ace6273 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148170 Aug 24 10:34:20.748524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1273/2411] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:20 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:21.185665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:21.186268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:21.192270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.192753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.193189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.193871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.194324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.194780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.321269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:21.329336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:21.330819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:21.331946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c4285c7c-fd9f-4bd8-83f0-482a9b788c31 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1599 microversion: 2.20 time: 0.151272 Aug 24 10:34:21.332528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1139/2412] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:34:21 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1599 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:34:21.355252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-99986370-a850-46e5-855e-1c8cfd386cbe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:21.355780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-99986370-a850-46e5-855e-1c8cfd386cbe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:21.356819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:21.357433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:21.362235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99986370-a850-46e5-855e-1c8cfd386cbe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.362719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99986370-a850-46e5-855e-1c8cfd386cbe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.363156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99986370-a850-46e5-855e-1c8cfd386cbe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.363276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.363733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.363860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99986370-a850-46e5-855e-1c8cfd386cbe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.364314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99986370-a850-46e5-855e-1c8cfd386cbe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.364431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.364741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99986370-a850-46e5-855e-1c8cfd386cbe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.365028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.365483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.365914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.444812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-99986370-a850-46e5-855e-1c8cfd386cbe tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.094373 Aug 24 10:34:21.445411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1274/2413] 10.222.0.86 () {60 vars in 1316 bytes} [Wed Aug 24 10:34:21 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936/os-interface => generated 285 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:34:21.493756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:21.502106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-61fcc1c4-cc5b-452a-a221-5a7f46a708b9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148984 Aug 24 10:34:21.502678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1140/2414] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:21 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:21.554954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:21.555573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:21.561741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.562224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.562655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.563344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.563795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.564246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.768985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:21.769902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:21.777292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.777781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.778273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.778974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:21.779432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:21.779861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:21.791205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:21.800368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3810df3c-e60b-435c-8bbf-418c3d668c33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.249695 Aug 24 10:34:21.801023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1275/2415] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:21 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:21.916739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:21.924984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7ff5c065-dc9c-42af-87c7-e646623b8c18 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.159844 Aug 24 10:34:21.925550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1141/2416] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:21 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:22.458016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-f12648c5-b25f-4aa5-94df-3b234c0ea0b0 req-a54b369e-4950-45dc-8c17-367ea3f90537 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9b34f307-a219-4d5f-aa5d-0e3eb79ea936", "tag": "b7060ba2-b694-4997-b4c4-21985557243b"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:34:22.468715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-f12648c5-b25f-4aa5-94df-3b234c0ea0b0 req-a54b369e-4950-45dc-8c17-367ea3f90537 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:22.469191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-f12648c5-b25f-4aa5-94df-3b234c0ea0b0 req-a54b369e-4950-45dc-8c17-367ea3f90537 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:22.469618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-f12648c5-b25f-4aa5-94df-3b234c0ea0b0 req-a54b369e-4950-45dc-8c17-367ea3f90537 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:22.480899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-f12648c5-b25f-4aa5-94df-3b234c0ea0b0 req-a54b369e-4950-45dc-8c17-367ea3f90537 service nova] Creating event network-changed:b7060ba2-b694-4997-b4c4-21985557243b for instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 on hv-cinder-85427 Aug 24 10:34:22.485869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-f12648c5-b25f-4aa5-94df-3b234c0ea0b0 req-a54b369e-4950-45dc-8c17-367ea3f90537 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031712 Aug 24 10:34:22.486445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1276/2417] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:34:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:34:22.522828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:22.524040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:22.526618 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:22.531279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:22.531766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:22.532249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:22.532946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:22.533399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:22.533821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:22.682426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:22.691097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-023e4c8c-1a82-440a-b700-e1b0b2ea9b05 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.171728 Aug 24 10:34:22.704399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1142/2418] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:22 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:22.823009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:22.823819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:22.832265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:22.832986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:22.833466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:22.834175 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:22.834635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:22.835131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:22.946876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:22.947494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:22.953492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:22.953984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:22.954424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:22.955115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:22.955568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:22.955994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:23.050351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:23.061670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f7f4ee6d-3202-45ba-a597-907c9c5e9db3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.243455 Aug 24 10:34:23.063138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1277/2419] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:22 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:23.098913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:23.108073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-47b8625e-042f-43aa-9442-e983796d245c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.165917 Aug 24 10:34:23.108902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1143/2420] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:22 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:23.712245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:23.712824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:23.715258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=1,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:23.721682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:23.722171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:23.722612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:23.723306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:23.723762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:23.724219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:23.857173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:23.865533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b59ad0a1-b548-4b39-a2d8-2ae27dafd173 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.157712 Aug 24 10:34:23.866103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1278/2421] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:23 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:24.083156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:24.083726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:24.090953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:24.091443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:24.091877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:24.092604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:24.093061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:24.093494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:24.129883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:24.130501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:24.137190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:24.137668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:24.138109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:24.138816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:24.139276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:24.139710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:24.227153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:24.235788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dc0fb86f-4f33-42c7-a045-6e8c894b5a42 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156509 Aug 24 10:34:24.236385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1144/2422] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:24 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:24.283444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:24.293474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1aa4f696-473d-4ab8-ae4b-3849a3f2a9ff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.167320 Aug 24 10:34:24.294050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1279/2423] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:24 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:24.886589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:24.887203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:24.893125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:24.893607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:24.894043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:24.894720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:24.895174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:24.895596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:25.022479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:25.031015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1b8ef18c-2e6e-4ab6-917a-7c87429f04ba tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148304 Aug 24 10:34:25.031580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1145/2424] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:24 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:25.257358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:25.258052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:25.264459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:25.264940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:25.265369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:25.266053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:25.266501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:25.266922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:25.314627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:25.315183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:25.321518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:25.321989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:25.322419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:25.323093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:25.323537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:25.323955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:25.394532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:25.403294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-87db1ace-a5ac-4933-bf40-0a375a1c0f1b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149750 Aug 24 10:34:25.403944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1280/2425] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:25 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:25.454855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:25.464353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b7d3056d-d2a6-4b89-92c9-2d2be8887a05 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153503 Aug 24 10:34:25.464929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1146/2426] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:25 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:26.052603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:26.053130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:26.060537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:26.060537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:26.060897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:26.061510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:26.061959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:26.062379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:26.195349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:26.203921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4a4ab81e-5b37-41e5-9d88-45057b1e95ab tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155220 Aug 24 10:34:26.204535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1281/2427] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:26 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:26.424855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:26.425426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:26.432969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:26.433462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:26.433900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:26.434600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:26.435054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:26.435512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:26.485466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:26.486182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:26.492599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:26.493091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:26.493531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:26.494894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:26.495366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:26.495797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:26.567130 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:26.575500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-86f3d916-56ef-4455-8296-440c13f5b2d4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154564 Aug 24 10:34:26.576295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1147/2428] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:26 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:26.630487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:26.638799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-58f65e7e-bf21-46be-ac14-de785fae5567 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.157207 Aug 24 10:34:26.639360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1282/2429] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:26 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:27.224832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:27.225444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:27.231704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:27.232208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:27.232640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:27.233312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:27.233765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:27.234181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:27.362727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:27.366087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:27.372225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3dff9754-7b42-4bc5-8620-c15379d53052 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151238 Aug 24 10:34:27.372785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1148/2430] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:27 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:27.597636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:27.598187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:27.605091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:27.605580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:27.606022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:27.606713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:27.607283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:27.607714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:27.661067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:27.661675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:27.668912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:27.670467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:27.670905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:27.671595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:27.672069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:27.672497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:27.736744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:27.745710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7a976d0d-d80c-4881-bd65-bbfe2ed1105c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151992 Aug 24 10:34:27.746273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1283/2431] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:27 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:27.802279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:27.811029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f28714f4-3e7f-48b9-8f78-5bd7dd507518 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153872 Aug 24 10:34:27.811588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1149/2432] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:27 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:28.393576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:28.394186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:28.400154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:28.400678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:28.401114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:28.401787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:28.402236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:28.402657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:28.533975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:28.542747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e6ead9a4-1885-4627-a53a-232af156a74e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153025 Aug 24 10:34:28.543313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1284/2433] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:28 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:28.766922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:28.767541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:28.773826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:28.774306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:28.774740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:28.775431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:28.776407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:28.776835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:28.832211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:28.832983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:28.840064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:28.840563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:28.841004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:28.841753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:28.842216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:28.842645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:28.910389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:28.919139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d52e8c67-e004-4668-988d-67a5abec5235 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156112 Aug 24 10:34:28.919933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1150/2434] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:28 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:28.974279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:28.983181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-72dd32fd-8c1e-4100-8818-9237e582d0fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154938 Aug 24 10:34:28.983772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1285/2435] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:28 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:29.563176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:29.563823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:29.569764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:29.570219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:29.570654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:29.571327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:29.571779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:29.572230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:29.699387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:29.707514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-13fb96ee-7075-4c1b-ae8e-7ee5c5acf1fe tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148216 Aug 24 10:34:29.708155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1151/2436] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:29 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:29.940270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:29.940889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:29.947363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:29.947851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:29.948320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:29.949013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:29.949469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:29.949933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:30.004994 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:30.005557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:30.011856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:30.012364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:30.012796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:30.013474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:30.013918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:30.014337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:30.078636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:30.086833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3a6910d5-f599-4a3a-b27d-a62b2181beac tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150478 Aug 24 10:34:30.087395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1286/2437] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:29 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:30.144078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:30.152243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1f0cbc58-e6d7-4172-a588-cfdb2a873f99 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151239 Aug 24 10:34:30.152805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1152/2438] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:29 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:30.728649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:30.729421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:30.736853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:30.737368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:30.737802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:30.738591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:30.739049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:30.739473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:30.868638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:30.876837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2033b9fb-1b51-463c-95b8-a859b786fff0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152068 Aug 24 10:34:30.877400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1287/2439] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:30 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:31.109097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:31.109906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:31.116354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:31.116839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:31.117276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:31.117959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:31.118416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:31.118840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:31.173190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:31.174067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:31.180341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:31.180833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:31.181272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:31.181969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:31.182428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:31.182855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:31.203167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:31.254779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:31.264441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-01ad2d5d-11e1-4a1a-91c2-4cc7374a82b2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.159442 Aug 24 10:34:31.265029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1153/2440] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:31 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:31.313948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:31.322064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-69662e5e-7c55-4bd7-abb6-4db589c28bae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152758 Aug 24 10:34:31.322622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1288/2441] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:31 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:31.897813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:31.898418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:31.904964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:31.905441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:31.905870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:31.906539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:31.906982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:31.907397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:32.032829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:32.041240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8d17a344-029f-4010-a167-7ba558b1383c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147276 Aug 24 10:34:32.041793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1154/2442] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:31 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:32.286026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:32.286586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:32.293068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:32.293554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:32.293996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:32.294685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:32.295144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:32.295572 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:32.345571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:32.346553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:32.355810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:32.356783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:32.357450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:32.358535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:32.359223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:32.359861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:32.438614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:32.447230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ed26a046-9a08-4d5d-a201-8d5ac12f8526 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.165193 Aug 24 10:34:32.447804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1289/2443] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:32 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:32.492017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:32.500650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0ba1021b-cf71-411c-8003-f4d0fadc0b6b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.161777 Aug 24 10:34:32.501215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1155/2444] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:32 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:33.063621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:33.064268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:33.070223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:33.070708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:33.071144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:33.071827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:33.072312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:33.072731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:33.201735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:33.210240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2f7acb67-b217-4d6a-b581-a44973221706 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150456 Aug 24 10:34:33.210810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1290/2445] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:33 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:33.469565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:33.470453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:33.477057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:33.477537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:33.477970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:33.478681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:33.479138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:33.479561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:33.522032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:33.522743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:33.529759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:33.530251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:33.530697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:33.531399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:33.531858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:33.532316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:33.610684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:33.618948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a46f7a81-c3f7-4f57-a3d6-4a6178f58dd9 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153689 Aug 24 10:34:33.619513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1156/2446] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:33 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:33.665332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:33.674902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-783d2753-081c-4c56-a34d-368497b85d14 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.156697 Aug 24 10:34:33.675475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1291/2447] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:33 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:34.230898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:34.231503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:34.237738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:34.238217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:34.238651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:34.239330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:34.239780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:34.240233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:34.379225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:34.388629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-44ef232e-05ae-483e-b261-1be5d8632143 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.161508 Aug 24 10:34:34.389205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1157/2448] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:34 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:34.640860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:34.641399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:34.647819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:34.648340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:34.648778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:34.649460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:34.649912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:34.650337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:34.695824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:34.696458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:34.702717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:34.703198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:34.703634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:34.704336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:34.704788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:34.705208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:34.780084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:34.789083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-76ab7983-c40d-48d6-a514-dd06b01e437e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152235 Aug 24 10:34:34.789746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1292/2449] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:34 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:34.839889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:34.848845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-172253a9-04ff-418d-bd57-0bb4f1e9f6f5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.156812 Aug 24 10:34:34.849408 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1158/2450] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:34 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:35.410191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:35.410809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:35.416964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:35.417445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:35.417886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:35.418574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:35.419033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:35.419459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:35.546036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:35.554202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ad51aa2b-66f6-49b5-85d1-ef6d6cb2142a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147898 Aug 24 10:34:35.554771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1293/2451] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:35 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:35.811462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:35.812187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:35.818614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:35.819094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:35.819533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:35.820251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:35.820705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:35.821125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:35.871317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:35.872667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:35.878610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:35.879207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:35.879709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:35.880690 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:35.881423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:35.881916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:35.957531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:35.966516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cfbf34e4-1f51-4b00-9a04-07124126f306 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.159508 Aug 24 10:34:35.967093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1159/2452] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:35 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:36.026926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:36.036487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a5801d8b-cb7c-4aaf-abed-95c98e754b25 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.169082 Aug 24 10:34:36.037192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1294/2453] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:35 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:36.577437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:36.578041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:36.584163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:36.584651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:36.585087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:36.585777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:36.586228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:36.586653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:36.716147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:36.724322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-03315a4d-cf0f-47e7-8af1-fab293b5a015 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152158 Aug 24 10:34:36.724961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1160/2454] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:36 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:36.989200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:36.989848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:36.996267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:36.996831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:36.997356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:36.998478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:36.998928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:36.999349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:37.058056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:37.059372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:37.065777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:37.066260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:37.066698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:37.067394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:37.067849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:37.068296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:37.127993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:37.136193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f429bf9b-f5d5-4bef-9daa-6bcff5f44090 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150855 Aug 24 10:34:37.136773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1295/2455] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:36 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:37.195814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:37.204190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6de0f1db-a13a-47bd-b873-a98f0e9c236b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149935 Aug 24 10:34:37.204756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1161/2456] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:37 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:37.744904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:37.745540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:37.752989 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:37.753473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:37.753909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:37.754593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:37.755048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:37.755476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:37.888257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:37.896989 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cd8a6862-8500-433c-85e1-a6e184eecb2c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155967 Aug 24 10:34:37.897547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1296/2457] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:37 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:38.157373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:38.157933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:38.164415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:38.164900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:38.165337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:38.166022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:38.166480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:38.166908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:38.225773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:38.226393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:38.233278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:38.234065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:38.234509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:38.235201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:38.235659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:38.236108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:38.299694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:38.308101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-15ceea8b-125d-4b27-8312-4440f47d5ec7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154655 Aug 24 10:34:38.308674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1162/2458] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:38 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:38.364718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:38.372776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f5e2f525-2217-4a4a-8515-a267f403d2fc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151027 Aug 24 10:34:38.374067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1297/2459] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:38 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:38.917801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:38.918417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:38.925008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:38.925488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:38.925922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:38.926606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:38.927061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:38.927481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:39.054619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:39.062696 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a8e845b2-8ce4-4c08-b180-8127b4d7e8f6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148824 Aug 24 10:34:39.063252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1163/2460] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:38 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:39.330455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:39.330999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:39.337355 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:39.337830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:39.338288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:39.338964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:39.339412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:39.339835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:39.395098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:39.395702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:39.401984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:39.402459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:39.402885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:39.403568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:39.404014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:39.404460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:39.472338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:39.480473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-888bd63d-f83d-4184-bc27-725b7c573fdc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154434 Aug 24 10:34:39.481042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1298/2461] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:39 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:39.539437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:39.547680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6f7241db-e230-4fe6-8b4b-b151d485c833 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.156373 Aug 24 10:34:39.548276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1164/2462] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:39 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:40.083148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:40.083857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:40.090509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:40.091001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:40.091443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:40.092178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:40.093130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:40.093587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:40.224890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:40.234132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bd4f5e74-b80d-433f-af90-39ffd561d28a tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154420 Aug 24 10:34:40.236685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1299/2463] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:40 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:40.501892 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:40.502529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:40.509000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:40.509472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:40.509899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:40.510567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:40.511016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:40.511427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:40.529422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:40.567804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:40.568478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:40.575018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:40.575502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:40.575937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:40.576657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:40.577604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:40.578029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:40.645643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:40.653829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-552eb8d3-7c5d-4804-9ec6-ee5e662c5d0e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155807 Aug 24 10:34:40.654390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1165/2464] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:40 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:40.706501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:40.714808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f8cc02cb-1588-40fb-b206-d668493f6708 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150880 Aug 24 10:34:40.715432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1300/2465] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:40 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:41.256060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:41.256687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:41.262651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:41.263129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:41.263559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:41.264278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:41.264713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:41.265132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:41.390024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:41.398162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8584735f-bad1-4f3b-884c-1e74dfc25588 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.146003 Aug 24 10:34:41.398727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1166/2466] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:41 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:41.676504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:41.677038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:41.683396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:41.683886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:41.684372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:41.685061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:41.685520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:41.685950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:41.736889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:41.737503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:41.743836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:41.744354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:41.744794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:41.745485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:41.745940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:41.746366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:41.818691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:41.827256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e9c8c2a5-cdd4-4434-9df2-5e30934e84b1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154405 Aug 24 10:34:41.828276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1301/2467] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:41 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:41.879770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:41.888033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3c74d207-0d42-4a78-b7d3-012c1d694051 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.155229 Aug 24 10:34:41.888634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1167/2468] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:41 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:42.419008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:42.419626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:42.426274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:42.426761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:42.427203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:42.427887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:42.428374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:42.428801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:42.553308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:42.561523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cdf36ea2-336a-4845-bc76-16c68d96d264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.146389 Aug 24 10:34:42.562093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1302/2469] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:42 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:42.849252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:42.849859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:42.856311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:42.856782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:42.857237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:42.857912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:42.858386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:42.858799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:42.910221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:42.910933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:42.917501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:42.918065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:42.918583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:42.919357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:42.919879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:42.920434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:42.986520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:42.994726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-eb508112-b863-4318-abad-443731e3aaa4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149311 Aug 24 10:34:42.995495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1168/2470] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:42 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:43.052614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:43.060784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-84301122-405c-4cad-a93b-0747d5bd19dd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.155020 Aug 24 10:34:43.061386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1303/2471] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:42 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:43.582572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:43.583185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:43.589211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:43.589694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:43.590128 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:43.590807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:43.591257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:43.591679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:43.717575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:43.725750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4b3377ec-b389-4f22-b749-6b618816848e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147030 Aug 24 10:34:43.726315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1169/2472] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:43 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:44.015805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:44.016462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:44.022802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:44.023286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:44.023727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:44.024452 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:44.024908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:44.025333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:44.081848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:44.082494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:44.088973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:44.089462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:44.089903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:44.090595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:44.091060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:44.091481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:44.165928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:44.176909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-989b26b0-0516-47af-8d77-dcf9eebb1a14 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.164894 Aug 24 10:34:44.177941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1304/2473] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:44 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:44.229791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:44.238322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-16ebd5f8-7aeb-4974-96e3-8beb86d73bad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.160786 Aug 24 10:34:44.238908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1170/2474] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:44 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:44.747386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:44.748001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:44.754448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:44.754928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:44.755366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:44.756123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:44.756554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:44.756978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:44.760879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:44.888181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:44.896913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-59161503-e11c-48fa-8a59-3d6039dedb0b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153374 Aug 24 10:34:44.897478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1305/2475] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:44 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:45.198252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:45.198901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:45.205452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:45.205933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:45.206396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:45.207081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:45.207534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:45.207962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:45.258861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:45.259479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:45.265856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:45.266352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:45.266794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:45.267491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:45.267961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:45.268435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:45.338483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:45.347073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-54c578da-a28c-49e1-9708-2dd441b0a2b4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152707 Aug 24 10:34:45.347644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1171/2476] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:45 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:45.399877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:45.408648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b444cbab-fd9e-4445-a854-cbac26558c8b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153665 Aug 24 10:34:45.409214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1306/2477] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:45 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:45.917515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:45.918123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:45.924117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:45.924593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:45.925018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:45.925694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:45.926141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:45.926553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:46.055990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:46.064619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b360af49-da2f-47b4-9e39-1e8dcf3dbb52 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150936 Aug 24 10:34:46.065174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1172/2478] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:45 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:46.368994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:46.369615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:46.376608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:46.377097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:46.377540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:46.378229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:46.378688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:46.379118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:46.430262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:46.430876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:46.437253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:46.437739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:46.438176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:46.439445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:46.439921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:46.440339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:46.509263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:46.518004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0802a107-ba5f-4ee5-b09c-a2a56af7e30f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152570 Aug 24 10:34:46.518679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1307/2479] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:46 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:46.570927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:46.579031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-86a63b30-414c-466e-a069-a212754c70a3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152745 Aug 24 10:34:46.579620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1173/2480] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:46 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:47.086027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:47.086611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:47.092585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:47.093070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:47.093510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:47.094193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:47.094695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:47.095584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:47.224904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:47.233010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d1a9f012-9b1c-4338-9de3-7f53b19b3844 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150927 Aug 24 10:34:47.233578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1308/2481] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:47 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:47.539231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:47.539880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:47.546290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:47.546771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:47.547208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:47.547891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:47.549009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:47.549535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:47.600581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:47.601201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:47.607542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:47.608028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:47.608496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:47.609193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:47.609657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:47.610299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:47.681949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:47.690138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3b40e5c6-debb-4a5d-9434-4593e739233c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154773 Aug 24 10:34:47.690708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1174/2482] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:47 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:47.738982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:47.747096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-510ebf98-8dd8-467a-8682-e9ed60b8f325 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150366 Aug 24 10:34:47.747661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1309/2483] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:47 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:48.253938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:48.254557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:48.260671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:48.261151 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:48.261586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:48.262264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:48.262718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:48.263139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:48.389585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:48.398348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-43ea477f-a1a3-4301-a85e-13474b25fab0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148309 Aug 24 10:34:48.398911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1175/2484] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:48 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:48.711034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:48.711656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:48.718116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:48.718599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:48.719035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:48.719726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:48.720210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:48.720635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:48.768125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:48.768744 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:48.775389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:48.775873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:48.776345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:48.777035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:48.777487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:48.777911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:48.852865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:48.861960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f6896b3e-966e-4795-b3ca-b68bfdc6db78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154287 Aug 24 10:34:48.862974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1310/2485] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:48 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:48.910855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:48.919013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ce9dfb97-d4f7-4703-9099-510adec4b2b6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154752 Aug 24 10:34:48.919580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1176/2486] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:48 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:49.418265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:49.418877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:49.425439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:49.425927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:49.426367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:49.427053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:49.427507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:49.427932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:49.560622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:49.569422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e4e76bd7-46a2-41ea-922f-5f1bd855751b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155116 Aug 24 10:34:49.570023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1311/2487] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:49 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:49.883472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:49.884121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:49.889947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:49.890421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:49.890850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:49.891526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:49.891990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:49.892426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:49.940129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:49.940714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:49.947794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:49.948306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:49.948739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:49.949421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:49.949868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:49.950292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:50.024892 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:50.033314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2b88a23d-f5a2-46f5-a210-41e0a945b4f5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153879 Aug 24 10:34:50.033885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1177/2488] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:49 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:50.080163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:50.089003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-60b8e959-c07d-4374-9ed5-83375ce61f1b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153027 Aug 24 10:34:50.089577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1312/2489] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:49 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:50.590308 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:50.590922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:50.597410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:50.597915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:50.598352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:50.599036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:50.599511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:50.599919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:50.672327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:34:50.672591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:34:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad (ip 10.222.0.86) !!! Aug 24 10:34:50.725793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:50.734099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b2475ee8-d88f-4732-bda7-501a0723e917 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147738 Aug 24 10:34:50.734665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1178/2490] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:50 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:51.054716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:51.055330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:51.061741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:51.062249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:51.062692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:51.063372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:51.063827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:51.064283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:51.109408 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:51.110020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:51.117235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:51.117714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:51.118140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:51.118825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:51.119271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:51.119692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:51.197930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:51.206242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-099dd87b-dc04-4455-a58f-6a9bbeb2f84c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155349 Aug 24 10:34:51.206822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1313/2491] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:51 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:51.254017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:51.262139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-527c5068-f6cc-4591-8988-949b1c42768b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.156577 Aug 24 10:34:51.262700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1179/2492] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:51 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:51.755673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:51.756273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:51.762270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:51.762757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:51.763232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:51.763931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:51.764427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:51.764862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:51.892602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:51.900703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7dbd1134-0a94-4e0d-b706-4547a2806ede tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149656 Aug 24 10:34:51.901272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1314/2493] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:51 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:52.228403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:52.229048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:52.235414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:52.235891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:52.236463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:52.237144 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:52.238230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:52.238652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:52.282796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:52.283422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:52.290215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:52.290697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:52.291136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:52.291828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:52.292319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:52.292750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:52.367683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:52.375942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-068f24dc-7d27-4488-8e99-e53ffb8e8206 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151326 Aug 24 10:34:52.376535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1180/2494] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:52 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:52.423505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:52.432357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f463044a-9737-49b5-9652-2f418cd7cfa6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153370 Aug 24 10:34:52.432919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1315/2495] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:52 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:52.921653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:52.922224 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:52.924348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=78 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:52.929477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:52.929959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:52.930397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:52.931078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:52.931533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:52.931963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:53.057526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:53.065546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-168231ea-4923-4241-8be2-2059e1d110a7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147833 Aug 24 10:34:53.066603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1181/2496] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:52 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:53.397057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:53.397666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:53.404160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:53.404653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:53.405092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:53.405786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:53.406245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:53.406672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:53.453211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:53.453831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:53.460973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:53.461467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:53.461905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:53.462596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:53.463052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:53.463507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:53.467452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:53.539562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:53.547823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5558178c-e6bc-4fc6-a582-75a5fcaef322 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154536 Aug 24 10:34:53.548419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1316/2497] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:53 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:53.599110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:53.607281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f005d259-512b-4f40-93fd-095e87f846fb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.157941 Aug 24 10:34:53.607846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1182/2498] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:53 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:54.087212 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:54.087752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:54.089618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=78 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:54.094721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:54.095207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:54.095646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:54.096373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:54.096826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:54.097861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:54.228381 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:54.236477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-84e0bc74-44f7-4212-982b-a957e9c8ef62 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153158 Aug 24 10:34:54.237316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1317/2499] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:54 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:54.577871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:54.578485 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:54.585906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:54.586382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:54.586814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:54.587496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:54.587948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:54.588400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:54.629904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:54.630531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:54.636993 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:54.637581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:54.638031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:54.638730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:54.639192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:54.639625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:54.717859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:54.726319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1eb41a0b-ae00-4d8f-b626-ba599c30e856 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152340 Aug 24 10:34:54.726902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1183/2500] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:54 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:54.772492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:54.780609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0676abf2-a1bc-47c5-9294-60a2f392692b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.155080 Aug 24 10:34:54.781172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1318/2501] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:54 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:55.257447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:55.258104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:55.264068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:55.264565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:55.264996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:55.265677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:55.266125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:55.266546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:55.394520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:55.402752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-56ac557a-8f55-4ae8-9624-3fa668d36669 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149510 Aug 24 10:34:55.403313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1184/2502] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:55 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:55.748369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:55.749015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:55.755442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:55.755930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:55.756404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:55.757093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:55.757547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:55.758007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:55.801905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:55.802501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:55.808877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:55.809357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:55.809791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:55.810476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:55.810928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:55.811557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:55.890686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:55.901593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-79eeeeb5-b175-4f9a-8534-ad912763ff8a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.157122 Aug 24 10:34:55.902172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1319/2503] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:55 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:55.941415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:55.950001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-01c77b6e-ed4d-4f65-934a-5e8590677f30 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152038 Aug 24 10:34:55.950552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1185/2504] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:55 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:56.423105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:56.423716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:56.430206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:56.430696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:56.431126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:56.431842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:56.432402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:56.432828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:56.562444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:56.570874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1e572598-b48d-40e6-9694-2c6e93d56ba7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151664 Aug 24 10:34:56.571447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1320/2505] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:56 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:56.922663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:56.923255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:56.930386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:56.930864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:56.931315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:56.931984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:56.932508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:56.932932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:56.971708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:56.972507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:56.979985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:56.980503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:56.980943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:56.981640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:56.982096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:56.982523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:57.063434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:57.071810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-49f4c335-3233-4f9e-aef3-b47f340172af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152962 Aug 24 10:34:57.072477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1186/2506] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:56 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:57.111022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:57.114402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:34:57.120102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0449727d-dfb4-4e21-bd63-f94efe04704d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152270 Aug 24 10:34:57.120668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1321/2507] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:56 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:57.591568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:57.592199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:57.598100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:57.598713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:57.599061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:57.599718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:57.600192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:57.600607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:57.727488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:57.735595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a4516a76-9807-4e30-ac90-e92208e8f286 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147889 Aug 24 10:34:57.736172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1187/2508] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:57 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:58.093939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:58.094478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:58.101577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:58.102068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:58.102513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:58.103209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:58.103670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:58.104128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:58.141415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:58.142152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:58.148995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:58.149561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:58.150175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:58.150924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:58.151379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:58.151815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:58.233567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:58.243057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b67ca355-9f62-4b52-aa80-f6bfccd9d37a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153093 Aug 24 10:34:58.243642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1322/2509] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:58 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:58.282495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:58.291215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a2aa39d6-4493-40ec-93c0-4f34807102c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153982 Aug 24 10:34:58.291778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1188/2510] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:58 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:58.653764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-791ba32e-5b08-4c2e-899d-5c734b1738d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:58.654290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-791ba32e-5b08-4c2e-899d-5c734b1738d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:58.660287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-791ba32e-5b08-4c2e-899d-5c734b1738d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:58.660750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-791ba32e-5b08-4c2e-899d-5c734b1738d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:58.661228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-791ba32e-5b08-4c2e-899d-5c734b1738d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:58.661911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-791ba32e-5b08-4c2e-899d-5c734b1738d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:58.662365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-791ba32e-5b08-4c2e-899d-5c734b1738d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:58.662791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-791ba32e-5b08-4c2e-899d-5c734b1738d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:58.685909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-791ba32e-5b08-4c2e-899d-5c734b1738d5 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936/ips" status: 200 len: 152 microversion: 2.20 time: 0.036041 Aug 24 10:34:58.686466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1323/2511] 10.222.0.86 () {60 vars in 1289 bytes} [Wed Aug 24 10:34:58 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936/ips => generated 152 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 24 10:34:58.757349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:58.757961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:58.764153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:58.764645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:58.765086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:58.765813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:58.766271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:58.766697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:58.898401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:58.906516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-32907974-6c18-4289-9efe-5c250e74e5e1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153590 Aug 24 10:34:58.907013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1189/2512] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:58 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:59.264375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:59.264999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:59.271908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:59.272433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:59.272875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:59.273569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:59.274024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:59.274453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:59.312815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:59.313429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:59.320446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:59.320937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:59.321371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:59.322070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:59.322524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:59.322951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:59.458785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:59.467424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-49bbf374-6a79-488c-9e4e-7d8fa8917c6d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.158347 Aug 24 10:34:59.468008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1190/2513] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:59 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:34:59.628344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:34:59.637300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a594d1e0-cbb9-408e-9c78-8038003b447a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.376922 Aug 24 10:34:59.637871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1324/2514] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:59 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:34:59.929111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:34:59.929637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:34:59.941767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:59.945246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:59.945689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:34:59.946399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:34:59.946853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:34:59.947276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:00.076933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:00.085027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9313bf95-e4dd-4d18-ad5a-23e3d2aea19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.161417 Aug 24 10:35:00.085587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1191/2515] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:34:59 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:00.488629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:00.489226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:00.496693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:00.497173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:00.497605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:00.498282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:00.498732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:00.499157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:00.624329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:00.632635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c792cb80-85bf-4747-842f-cae2938ec901 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.147577 Aug 24 10:35:00.632960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1325/2516] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:00 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:00.659833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:00.660481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:00.667364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:00.667833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:00.668302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:00.668974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:00.669417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:00.669833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:00.798515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:00.806544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-725fdea2-a36c-462e-90e4-78ccc9abe9a1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150529 Aug 24 10:35:00.807097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1192/2517] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:00 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:01.105528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:01.106068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:01.112066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:01.112566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:01.113007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:01.113698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:01.114163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:01.114626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:01.246918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:01.255038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5f649bb9-616a-4d3b-9ec2-a1a1f2fd48ad tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153503 Aug 24 10:35:01.255595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1326/2518] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:01 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:01.653459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:01.654063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:01.660152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:01.660743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:01.661047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:01.661717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:01.662821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:01.663239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:01.791419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:01.799633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f055d908-b808-445d-815c-6193a8b8c33e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149989 Aug 24 10:35:01.800229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1193/2519] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:01 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:01.829104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:01.829715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:01.836816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:01.837295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:01.837729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:01.838410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:01.838859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:01.839312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:01.965310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:01.973477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e33f68c6-71d9-421a-b3b6-dfc78c60a8db tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148230 Aug 24 10:35:01.974040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1327/2520] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:01 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:02.276461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:02.277067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:02.283238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:02.283747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:02.284225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:02.284914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:02.285369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:02.285800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:02.415775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:02.423866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d1d77dfa-ca8d-4153-bafc-b896ed70d19f tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151227 Aug 24 10:35:02.424462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1194/2521] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:02 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:02.820873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:02.821481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:02.827366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:02.827846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:02.828322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:02.828996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:02.829447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:02.829988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:02.960761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:02.968807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f263ae32-0a61-4353-b331-1ff8269abc56 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151808 Aug 24 10:35:02.969359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1328/2522] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:02 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:02.994983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:02.995591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:03.001925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:03.002404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:03.002838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:03.003511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:03.003962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:03.004407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:03.129751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:03.137816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-53541153-e7a0-4de0-9106-03f0b7e118b6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.146964 Aug 24 10:35:03.138368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1195/2523] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:02 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:03.445992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:03.446608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:03.455593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:03.456203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:03.456646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:03.457339 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:03.457790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:03.458241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:03.587437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:03.595720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-51a01df5-11a8-4e67-b338-d36367dc0131 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154409 Aug 24 10:35:03.596385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1329/2524] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:03 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:03.990507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:03.991123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:03.997578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:03.998162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:03.998488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:03.999172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:03.999624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:04.000065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:04.129525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:04.138216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e3a072c7-4add-433f-a0fe-8c062f34b389 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151646 Aug 24 10:35:04.138784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1196/2525] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:03 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:04.158549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:04.159172 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:04.165569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:04.166055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:04.166500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:04.167188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:04.167649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:04.168113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:04.299625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:04.308017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c3e04cfb-c351-44d1-8189-6532b9b5b5bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153216 Aug 24 10:35:04.308672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1330/2526] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:04 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:04.617675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:04.618532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:04.624351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:04.624829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:04.625262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:04.625930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:04.626374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:04.626788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:04.754969 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:04.764120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6efd7b7d-c3fb-4689-a271-6c81b73a1576 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150849 Aug 24 10:35:04.764693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1197/2527] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:04 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:05.160314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:05.160952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:05.167856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:05.168366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:05.168802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:05.169485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:05.169932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:05.170355 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:05.298246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:05.306343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7630389f-9a0f-44c6-9f20-30f3126dd98e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149953 Aug 24 10:35:05.306895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1331/2528] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:05 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:05.329992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:05.330603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:05.337557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:05.338039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:05.338480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:05.339195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:05.339646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:05.340110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:05.471033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:05.474437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:05.481046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d20a3db0-554d-4621-9bb0-b1ee6d4f49cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154824 Aug 24 10:35:05.481614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1198/2529] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:05 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:05.785009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:05.785623 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:05.791589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:05.792129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:05.792567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:05.793247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:05.793695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:05.794113 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:05.980153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:05.988374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fa7a1205-6cd4-45d6-9331-aeb3964f0ed3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.207229 Aug 24 10:35:05.988931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1332/2530] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:05 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:06.327574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:06.328223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:06.334551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:06.335164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:06.335445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:06.336137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:06.336591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:06.337006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:06.466276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:06.474400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0c61ae9f-8683-4fa5-9493-c5733d71adfa tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150738 Aug 24 10:35:06.475024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1199/2531] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:06 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:06.502362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:06.502985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:06.510051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:06.510538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:06.510986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:06.511682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:06.512167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:06.512600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:06.642185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:06.650675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-54f0df2e-7591-4c27-9737-07a79bb73a7d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151869 Aug 24 10:35:06.651259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1333/2532] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:06 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:07.117928 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:07.118539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:07.124757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:07.125236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:07.125699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:07.126382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:07.126830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:07.127250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:07.306221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:07.314359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c4688919-8024-4b02-8998-44a961847589 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.308271 Aug 24 10:35:07.314920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1200/2533] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:07 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:07.496549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:07.497190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:07.503108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:07.503647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:07.504033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:07.504767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:07.505217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:07.505637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:07.633897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:07.642100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e99351f9-cec4-4bff-a9e2-6cac2cff494e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150208 Aug 24 10:35:07.642666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1334/2534] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:07 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:07.803039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:07.803662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:07.810982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:07.811524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:07.811898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:07.812689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:07.813235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:07.813667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:07.942832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:07.950892 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f09ecc93-c888-4443-8e03-df8b0e070969 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.282257 Aug 24 10:35:07.951446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1201/2535] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:07 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:08.335439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:08.336580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:08.342605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:08.343089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:08.343538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:08.344251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:08.344707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:08.345136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:08.481312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:08.489448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-31df11ad-b6d6-4eb3-80e7-4dd70cba9629 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.157981 Aug 24 10:35:08.490002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1335/2536] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:08 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:08.663348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:08.663870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:08.669851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:08.670328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:08.670753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:08.671428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:08.671875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:08.672324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:08.801964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:08.810155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-838fb51e-e9ba-4822-87a3-62db62700ec9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150830 Aug 24 10:35:08.810739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1202/2537] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:08.972168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:08.972783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:08.979210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:08.979688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:08.980154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:08.980833 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:08.981284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:08.981702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:09.110924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:09.119795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bb8b2959-b48c-48e6-880c-0e510f61eaba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151245 Aug 24 10:35:09.120392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1336/2538] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:08 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:09.510648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:09.511259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:09.517413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:09.517896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:09.518332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:09.519034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:09.519502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:09.519907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:09.647614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:09.656009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-01b8c2f9-6a3c-40d5-817a-413753462d3b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149496 Aug 24 10:35:09.656601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1203/2539] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:09 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:09.830916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:09.831534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:09.838026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:09.838514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:09.838959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:09.839654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:09.840142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:09.840575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:09.968697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:09.978090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-750df395-5362-4108-82ff-5a4b0bd07573 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151011 Aug 24 10:35:09.979046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1337/2540] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:09 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:10.141657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:10.142342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:10.149703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:10.150207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:10.150643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:10.151329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:10.151777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:10.152227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:10.279714 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:10.287896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a223df74-2b1c-4c24-bc71-8b09b468af3c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150152 Aug 24 10:35:10.288491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1204/2541] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:10 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:10.677405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:10.678044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:10.684646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:10.685131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:10.685567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:10.686246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:10.686698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:10.687123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:10.829685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:10.833160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:10.838907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-34fc43a3-e4b6-40a1-9432-aa41d55c7c57 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.165415 Aug 24 10:35:10.839481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1338/2542] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:10 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:10.999577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:11.000216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:11.006210 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:11.006694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:11.007136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:11.007813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:11.008293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:11.008719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:11.139940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:11.148637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3ca23d75-48e2-4190-8e68-fecae5499977 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152839 Aug 24 10:35:11.149584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1205/2543] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:10 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:11.309926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:11.310473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:11.317106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:11.317598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:11.318045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:11.318747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:11.319206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:11.319635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:11.457816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:11.466365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-120e33db-2031-4627-8f95-8a6efeaa228e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.160988 Aug 24 10:35:11.466949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1339/2544] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:11 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:11.859674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:11.860309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:11.866327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:11.866835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:11.867273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:11.867955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:11.868983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:11.869403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:11.998082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:12.006208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d2f7b41f-2fa0-461e-a42a-7bd1464d4624 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150378 Aug 24 10:35:12.006769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1206/2545] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:11 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:12.170458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:12.170984 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:12.177111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:12.177602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:12.178047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:12.178795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:12.179253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:12.179681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:12.310106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:12.318316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d1d98518-3ab0-4d49-8abb-65e0088d0196 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151849 Aug 24 10:35:12.318870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1340/2546] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:12.486804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:12.487440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:12.493817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:12.494285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:12.494713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:12.495637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:12.496133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:12.496549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:12.623211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:12.631332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ccd8b728-4ceb-4d16-bcd4-6c8a0c736912 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148392 Aug 24 10:35:12.631890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1207/2547] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:12 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:13.027582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:13.028161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:13.034091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:13.034572 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:13.035014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:13.035694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:13.036180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:13.036609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:13.166780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:13.175022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-62d94a41-ad11-4af0-84b7-c41a3b615248 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151363 Aug 24 10:35:13.175591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1341/2548] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:13 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:13.338789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:13.339313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:13.345234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:13.345705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:13.346133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:13.346798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:13.347240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:13.347657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:13.481334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:13.489644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b4a6c8d6-729f-44da-afa0-d58a65762c6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154837 Aug 24 10:35:13.490240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1208/2549] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:13 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:13.653199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:13.653787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:13.660219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:13.660707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:13.661143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:13.661823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:13.662278 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:13.662703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:13.793787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:13.802002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b4e43b6e-5628-46a5-aee3-a728b5811602 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153319 Aug 24 10:35:13.802569 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1342/2550] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:13 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:14.196283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:14.197649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:14.204369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:14.204822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:14.205283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:14.206063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:14.206518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:14.206942 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:14.333189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:14.341363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-30a96a90-65be-4124-9716-ae6ca480dc20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.148952 Aug 24 10:35:14.341922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1209/2551] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:14 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:14.511728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:14.512289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:14.518187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:14.518670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:14.519102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:14.519784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:14.520260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:14.520680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:14.649626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:14.657858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-aaf277f0-7a84-4dd1-b38f-283017801ed5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151043 Aug 24 10:35:14.658449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1343/2552] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:14 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:14.823292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:14.823903 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:14.829850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:14.830329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:14.830753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:14.831435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:14.831878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:14.832577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:14.961674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:14.970378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9207603b-13f8-4408-8545-5ee1278b068c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150962 Aug 24 10:35:14.970932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1210/2553] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:14 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:15.362484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:15.363091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:15.369546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:15.370030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:15.370465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:15.371148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:15.371855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:15.372304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:15.510956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:15.519190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d83bad3d-0516-4a04-bac6-8fac1ab9392e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.160672 Aug 24 10:35:15.519751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1344/2554] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:15 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:15.678688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:15.679226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:15.686102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:15.686580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:15.687018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:15.687698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:15.688178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:15.688601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:15.817567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:15.826419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dce1029a-5c84-4c9c-8ed9-ef643bc5675f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151645 Aug 24 10:35:15.827028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1211/2555] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:15 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:15.991946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:15.992593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:15.999313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:15.999779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:16.000265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:16.000950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:16.001405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:16.001831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:16.135528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:16.143739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b300e896-9652-4d42-9661-092e15faaf21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155674 Aug 24 10:35:16.144337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1345/2556] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:15 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:16.540372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:16.541003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:16.547763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:16.548286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:16.548775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:16.549470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:16.549923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:16.550350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:16.675493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:16.684171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6cc890c3-19f8-4f3c-b45e-1a2b881abb48 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147680 Aug 24 10:35:16.684734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1212/2557] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:16 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:16.846650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:16.847301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:16.853656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:16.854130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:16.854562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:16.855235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:16.855682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:16.856130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:16.981418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:16.989648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-04b9bd93-9f48-477f-bc48-ab67a5651e9c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.146862 Aug 24 10:35:16.990203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1346/2558] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:16 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:17.164721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:17.165363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:17.171324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:17.171806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:17.172344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:17.173027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:17.173506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:17.173929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:17.303445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:17.312152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-33001ac3-cadf-4630-96cd-86b3585d75ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151236 Aug 24 10:35:17.312716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1213/2559] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:17 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:17.705080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:17.705694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:17.711670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:17.712182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:17.712626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:17.713309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:17.713762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:17.714188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:17.845927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:17.855092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4ca6eddf-658a-4111-8093-2a5379dabf30 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153872 Aug 24 10:35:17.855661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1347/2560] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:17 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:18.011228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:18.011854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:18.018229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:18.018708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:18.019141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:18.019823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:18.020309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:18.020785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:18.148397 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:18.157043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-77373a23-821b-4594-a5fd-be372c8f4e8e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150265 Aug 24 10:35:18.157602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1214/2561] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:18 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:18.333777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:18.334322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:18.340379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:18.340881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:18.341346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:18.342035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:18.342501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:18.342931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:18.469889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:18.478097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b2ab14d9-fbdc-41c8-ae24-d77d8fb09c47 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148237 Aug 24 10:35:18.478658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1348/2562] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:18 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:18.876233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:18.876845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:18.882751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:18.883238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:18.883663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:18.884372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:18.884825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:18.885246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:18.902732 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:19.014375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:19.023077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c37cd388-8290-49b0-bcf8-341c2379a9b7 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150975 Aug 24 10:35:19.023584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1215/2563] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:18 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:19.178263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:19.178889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:19.186109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:19.186592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:19.187032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:19.187714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:19.188198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:19.188627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:19.316519 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:19.324708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-61120f6c-18dc-4eb5-9a29-65b7e39500da tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150330 Aug 24 10:35:19.325277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1349/2564] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:19 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:19.498973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:19.499588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:19.505703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:19.506180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:19.506613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:19.507288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:19.507742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:19.508190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:19.633491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:19.642235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9eb5d36d-e6d1-447a-8c63-d1e7945ecd20 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147390 Aug 24 10:35:19.642806 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1216/2565] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:19 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:20.044918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:20.045611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:20.051692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:20.052206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:20.052641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:20.053326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:20.053775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:20.054201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:20.184709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:20.193420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ac6b3981-f650-43c1-a693-971e3844e6e6 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153642 Aug 24 10:35:20.193980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1350/2566] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:20 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:20.345250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:20.345809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:20.351827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:20.352344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:20.352783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:20.353462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:20.353917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:20.354344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:20.480544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:20.489554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-98f0594b-ee23-44a3-9ebb-e742a9100e69 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148228 Aug 24 10:35:20.490126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1217/2567] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:20 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:20.663511 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:20.664038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:20.669974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:20.670462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:20.670910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:20.671599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:20.672070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:20.672510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:20.801665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:20.810129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-885d100d-5889-483f-b238-52482a48951e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150621 Aug 24 10:35:20.810700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1351/2568] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:20 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:21.214150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:21.214760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:21.221257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:21.221736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:21.222172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:21.222852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:21.223300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:21.223720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:21.351161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:21.360434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-70b111cb-56e7-482f-978b-56f25e9f48a5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150145 Aug 24 10:35:21.361001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1218/2569] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:21 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:21.510747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:21.511545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:21.518028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:21.518518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:21.518961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:21.519651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:21.520137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:21.520568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:21.651457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:21.659673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e3e06b22-7a81-4c0f-bf1c-f76c57afede1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152836 Aug 24 10:35:21.660259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1352/2570] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:21 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:21.830376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:21.830990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:21.836900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:21.837384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:21.837827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:21.838510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:21.839304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:21.839461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:21.967123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:21.975291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-32108a71-ee21-4861-a37b-873cd0bc5514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148894 Aug 24 10:35:21.975853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1219/2571] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:21 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:22.381842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:22.382459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:22.388946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:22.389432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:22.389875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:22.390565 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:22.391022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:22.391471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:22.519076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:22.527343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-06fe43af-08cf-43c6-9b80-63aed978fee4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149277 Aug 24 10:35:22.527939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1353/2572] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:22 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:22.680980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:22.681593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:22.688503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:22.689081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:22.689511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:22.690191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:22.690638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:22.691059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:22.997320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:22.997884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:23.004512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:23.005001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:23.005444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:23.006136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:23.006591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:23.007017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:23.062905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:23.071283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dab8bb59-783e-4798-9b23-fd0e5dc365b9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.394074 Aug 24 10:35:23.071856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1220/2573] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:22 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:23.139581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:23.147768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ee826deb-1045-48a2-86b1-945202a63d36 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155173 Aug 24 10:35:23.148362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1354/2574] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:22 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:23.550026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:23.550646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:23.552510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=80 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:23.557737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:23.558252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:23.558709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:23.559368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:23.559823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:23.560275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:23.684152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:23.692833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-56b6fc05-fcef-4632-83e0-695cff7c077c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.146696 Aug 24 10:35:23.693400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1221/2575] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:23 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:24.092811 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:24.093377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:24.099805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:24.100327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:24.100778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:24.102031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:24.102496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:24.102926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:24.121876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:24.169500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:24.170114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:24.177334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:24.177826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:24.178263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:24.178966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:24.179422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:24.179849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:24.240540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:24.249949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8406d4e1-423f-420a-b653-4f3c86ae853f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.161051 Aug 24 10:35:24.250534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1355/2576] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:24 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:24.317163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:24.325367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f67a753a-77f7-4a08-bf77-23ae90da6661 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.159796 Aug 24 10:35:24.325930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1222/2577] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:24 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:24.714137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:24.714746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:24.716592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=78 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:24.721475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:24.721951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:24.722396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:24.723080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:24.723531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:24.723951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:24.853837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:24.863098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-19dae6b2-36d7-496a-a586-97292a92c264 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152744 Aug 24 10:35:24.863684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1356/2578] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:24 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:25.272370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:25.273014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:25.279379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:25.279861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:25.280334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:25.281021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:25.281476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:25.281904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:25.345741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:25.346375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:25.352737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:25.353222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:25.353662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:25.354353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:25.354807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:25.355237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:25.409340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:25.417812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-026efeb5-f484-4bd2-a96f-9dde8f3ed617 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149427 Aug 24 10:35:25.418391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1223/2579] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:25 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:25.487847 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:25.495873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ba2ed7c0-8b8d-4a85-8711-21f85d677e33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153927 Aug 24 10:35:25.496483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1357/2580] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:25 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:25.884077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:25.884696 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:25.899403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:25.899403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:25.899403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:25.899403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:25.899957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:25.899957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:26.025692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:26.034666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a6c1f203-507e-471e-a638-bc1aff4155be tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154460 Aug 24 10:35:26.035230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1224/2581] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:25 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:26.439634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:26.440298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:26.446744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:26.447221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:26.447654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:26.448637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:26.449091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:26.449516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:26.517987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:26.518649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:26.525246 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:26.525734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:26.526173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:26.526872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:26.527327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:26.527870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:26.580099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:26.588649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f96ea5b6-1d5d-4ad7-be06-b771e8482de7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152703 Aug 24 10:35:26.589247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1358/2582] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:26 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:26.657615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:26.665768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9731708b-3122-4f5c-a796-90845955c468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151610 Aug 24 10:35:26.666275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1225/2583] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:26 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:27.054713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:27.055359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:27.061800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:27.062288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:27.062728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:27.063426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:27.063884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:27.064341 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:27.193865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:27.202715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7f666eaa-b8e3-4183-90fc-df63cbdd7c1e tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151918 Aug 24 10:35:27.203279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1359/2584] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:27 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:27.610463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:27.611174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:27.617652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:27.618142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:27.618591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:27.619279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:27.619737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:27.620192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:27.687370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:27.688101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:27.694673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:27.695233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:27.696033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:27.696365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:27.696817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:27.697242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:27.747933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:27.756435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d52ea6d4-ec99-45c0-aca8-9d64bbfbfcbb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150298 Aug 24 10:35:27.757003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1226/2585] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:27 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:27.828751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:27.836905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7e1c5a41-aebe-455c-8179-61e63cf81089 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154093 Aug 24 10:35:27.837469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1360/2586] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:27 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:28.223630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:28.224271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:28.230241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:28.230719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:28.231159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:28.231835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:28.232311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:28.232738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:28.361116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:28.369223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b3322062-5081-429d-a834-af267e0b9845 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149518 Aug 24 10:35:28.369791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1227/2587] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:28 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:28.777628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:28.778249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:28.785233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:28.785722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:28.786168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:28.786865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:28.787320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:28.787749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:28.858834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:28.859494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:28.866328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:28.866822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:28.867261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:28.867961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:28.868442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:28.868874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:28.918504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:28.927067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1d30ccc7-356f-407b-8736-c4a65eb8b605 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153299 Aug 24 10:35:28.927641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1361/2588] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:28 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:28.996829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:29.005409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-63084dc8-9d35-48de-83b3-14cdd772ba53 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150861 Aug 24 10:35:29.005967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1228/2589] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:28 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:29.391663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:29.392299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:29.398129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:29.398610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:29.399043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:29.399724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:29.400203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:29.400625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:29.529446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:29.537856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a25a2ba9-9b9d-4dd9-8d48-d9c0895202d3 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150673 Aug 24 10:35:29.538417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1362/2590] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:29 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:29.947559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:29.948202 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:29.955279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:29.955753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:29.956212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:29.956889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:29.957335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:29.957754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:30.026935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:30.027521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:30.033965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:30.034529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:30.034987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:30.035650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:30.036253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:30.036680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:30.088660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:30.096926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0d33c296-1b12-4c7f-89ce-29b6015a14eb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153121 Aug 24 10:35:30.097482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1229/2591] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:29 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:30.166901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:30.175158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b2480249-405b-463a-902a-33490184f580 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152235 Aug 24 10:35:30.175722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1363/2592] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:30 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:30.559215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:30.559833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:30.566859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:30.567355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:30.567792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:30.568524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:30.568979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:30.569407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:30.696272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:30.704370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bd535e29-d753-48e6-89bf-b61a44314637 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149014 Aug 24 10:35:30.704934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1230/2593] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:30 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:31.117575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:31.118215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:31.125495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:31.126227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:31.126662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:31.127337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:31.127787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:31.128239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:31.195893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:31.196619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:31.203073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:31.203552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:31.203980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:31.204693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:31.205136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:31.205576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:31.260501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:31.268777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ae2fefc0-de76-4a51-b7c7-de06f5239339 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154996 Aug 24 10:35:31.269345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1364/2594] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:31 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:31.337573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:31.345758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e33b6cf0-f8cd-4804-9a2e-6dc9d4b01f70 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153700 Aug 24 10:35:31.346319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1231/2595] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:31 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:31.725651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:31.726290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:31.732276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:31.732755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:31.733189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:31.733865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:31.734311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:31.734758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:31.861065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:31.869270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2abf8c81-3f0e-4e9f-aba8-f55d63935a60 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147445 Aug 24 10:35:31.869838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1365/2596] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:31 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:32.289808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:32.290418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:32.296918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:32.297400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:32.297830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:32.298512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:32.298960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:32.299384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:32.303353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:32.366948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:32.367650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:32.374742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:32.375319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:32.375830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:32.376895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:32.377427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:32.377918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:32.432122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:32.440863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-96daf831-f107-47cc-9780-a0c6aee1924c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154840 Aug 24 10:35:32.441427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1232/2597] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:32 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:32.509940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:32.519040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c2f59dc9-6aec-4c4a-bfcc-75a7a58bdf4f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155993 Aug 24 10:35:32.519607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1366/2598] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:32 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:32.891184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:32.891788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:32.897725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:32.898204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:32.898642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:32.899597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:32.900073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:32.900502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:33.031617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:33.039765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-55e96727-25d7-4844-88f4-f9ec08f3af53 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152977 Aug 24 10:35:33.040363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1233/2599] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:32 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:33.462738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:33.463361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:33.469906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:33.470392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:33.470834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:33.471525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:33.471982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:33.472439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:33.540688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:33.541405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:33.548282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:33.548758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:33.549191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:33.549867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:33.550313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:33.550950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:33.602956 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:33.612122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b1b29ad9-0321-4f22-81f0-481351588780 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153217 Aug 24 10:35:33.612740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1367/2600] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:33 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:33.681185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:33.690135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a10bdc84-b825-4805-93ff-7d1a2cc9a1eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153220 Aug 24 10:35:33.690702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1234/2601] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:33 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:34.061312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:34.061544 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:34.067334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:34.067807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:34.068346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:34.069022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:34.069467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:34.069885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:34.195985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:34.206665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-be799874-e191-423e-9450-0609819e2c1c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149571 Aug 24 10:35:34.207270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1368/2602] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:34 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:34.633882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:34.634516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:34.641153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:34.641642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:34.642078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:34.642799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:34.643255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:34.643680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:34.710767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:34.711388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:34.718402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:34.718886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:34.719324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:34.720003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:34.720478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:34.720900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:34.773804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:34.782833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-57335348-33ce-4ad0-8a66-a6597ed3b530 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152845 Aug 24 10:35:34.783481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1235/2603] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:34 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:34.859296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:34.868988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3aab0310-091b-4e8f-a052-d3b5129776ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.162056 Aug 24 10:35:34.869637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1369/2604] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:34 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:35.229243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:35.229879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:35.236575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:35.237052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:35.237479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:35.238154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:35.238623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:35.239042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:35.524447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:35.533178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-404a3cf4-0c56-4467-89f4-843d4aa241ae tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.307784 Aug 24 10:35:35.533735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1236/2605] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:35 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:35.804852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:35.805458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:35.811388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:35.811869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:35.812343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:35.813017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:35.813467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:35.813891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:35.889489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:35.891060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:35.897139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:35.897682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:35.898137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:35.898809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:35.899287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:35.899709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:35.946857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:35.961337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-848a3107-9bc5-4fe9-ba2c-7fa79a2803cd tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.160396 Aug 24 10:35:35.961925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1370/2606] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:35 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:36.047823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:36.056022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-934401ed-0396-41eb-9cd4-d43f601f89a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.170342 Aug 24 10:35:36.056601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1237/2607] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:35 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:36.554698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:36.555309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:36.561281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:36.561767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:36.562856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:36.563534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:36.563988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:36.564517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:36.694580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:36.702609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-771987a4-d8a3-48a8-83e3-19bdc0786a20 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151794 Aug 24 10:35:36.703167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1371/2608] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:36 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:36.982632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:36.983227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:36.990345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:36.990830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:36.991269 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:36.991958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:36.992441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:36.992867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:37.083280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:37.083923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:37.090422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:37.090885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:37.091321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:37.092024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:37.092483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:37.092902 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:37.097374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:37.124156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:37.132813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-01a8f482-90a9-4300-bb78-b14979c390ef tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154107 Aug 24 10:35:37.133379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1238/2609] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:36 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:37.223658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:37.231954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1081f026-f671-4dc6-9080-ba94a0dd502a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152809 Aug 24 10:35:37.232564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1372/2610] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:37 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:37.723394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:37.724014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:37.729965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:37.730438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:37.730873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:37.731545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:37.731991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:37.732434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:37.863376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:37.871740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c5d61b46-e753-4a51-b578-7d22513b53c8 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152206 Aug 24 10:35:37.872400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1239/2611] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:37 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:38.154432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:38.155018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:38.162121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:38.162602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:38.163035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:38.163718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:38.164194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:38.164616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:38.253908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:38.254525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:38.261228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:38.261709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:38.262147 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:38.262840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:38.263297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:38.263722 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:38.295158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:38.304454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2069f763-a315-4a1b-b8e9-9933d47070be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153837 Aug 24 10:35:38.305167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1373/2612] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:38 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:38.396279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:38.404500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-80497ad7-5515-4dd3-8956-1fde8ddf1949 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154574 Aug 24 10:35:38.404995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1240/2613] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:38 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:38.893164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:38.893806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:38.899627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:38.900210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:38.900843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:38.901529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:38.901986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:38.902406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:39.066226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:39.075695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7ae59f25-833f-48b4-a4eb-781f39805d27 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.186841 Aug 24 10:35:39.076316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1374/2614] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:38 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:39.325726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:39.326285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:39.332711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:39.333195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:39.333627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:39.334427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:39.334758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:39.335183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:39.426955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:39.427684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:39.434045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:39.434749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:39.435192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:39.435886 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:39.436367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:39.436875 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:39.466190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:39.474747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ca1b3238-4c78-4c0b-8978-a7fda4697ed3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152951 Aug 24 10:35:39.475310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1241/2615] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:39 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:39.566287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:39.574657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-075b4f1d-3475-48e5-820e-453178ace7ef tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152103 Aug 24 10:35:39.575227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1375/2616] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:39 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:40.098119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:40.098652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:40.104628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:40.105111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:40.105548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:40.106235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:40.106687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:40.107115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:40.239686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:40.249126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3cd301f7-1334-43d6-83f4-5637a5cf3776 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155636 Aug 24 10:35:40.249764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1242/2617] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:40 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:40.496800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:40.497422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:40.504687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:40.505180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:40.505627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:40.506320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:40.506810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:40.507248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:40.595604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:40.596276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:40.602685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:40.603175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:40.603613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:40.604336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:40.604794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:40.605247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:40.638681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:40.648432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-34a6c703-435e-44dc-b4dc-00959e272498 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.155387 Aug 24 10:35:40.649261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1376/2618] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:40 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:40.734074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:40.741976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b6bcf46a-b710-47f4-9baf-3758accbfd6e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150241 Aug 24 10:35:40.742537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1243/2619] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:40 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:41.270770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:41.271388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:41.277446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:41.277937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:41.278382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:41.279067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:41.279523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:41.279952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:41.408901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:41.417107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c7d22755-f7bd-4f6d-a22f-34b8c21c87d5 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150238 Aug 24 10:35:41.417678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1377/2620] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:41 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:41.668912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:41.669507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:41.675961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:41.676479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:41.676920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:41.677606 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:41.678062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:41.678490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:41.764000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:41.764711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:41.776700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:41.777197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:41.777677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:41.778378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:41.778839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:41.779270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:41.825668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:41.834310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d48a1b6a-9a89-43a8-988c-bf25adb48ceb tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.169285 Aug 24 10:35:41.834871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1244/2621] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:41 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:41.914065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:41.922393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-688e0be7-24fd-4644-913b-7a2b46c03131 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.162246 Aug 24 10:35:41.922899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1378/2622] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:41 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:42.438684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:42.439904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:42.445875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:42.446350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:42.446778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:42.447451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:42.447894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:42.448341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:42.573771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:42.581868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a8ccd041-7070-4c61-936d-857c548cc49b tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147126 Aug 24 10:35:42.582427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1245/2623] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:42 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:42.856395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:42.857015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:42.864210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:42.864703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:42.865146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:42.865843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:42.866302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:42.866729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:42.947134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:42.947754 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:42.954323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:42.954805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:42.955247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:42.955939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:42.956421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:42.956873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:42.997492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:43.006161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-414f7c67-7b79-42bb-999f-b449f16fa1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153610 Aug 24 10:35:43.006741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1379/2624] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:42 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:43.088572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:43.097402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8d81f99d-d79d-44e5-a3ae-cdf47f8889ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154513 Aug 24 10:35:43.098190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1246/2625] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:42 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:43.602523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:43.603140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:43.609167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:43.609647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:43.610079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:43.610754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:43.611202 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:43.611625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:43.741456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:43.749636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-23301e28-4a7a-4bd3-8e92-a194bd7c6034 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151047 Aug 24 10:35:43.750138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1380/2626] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:43 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:44.027612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:44.028203 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:44.034553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:44.035035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:44.035480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:44.036231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:44.036690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:44.037110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:44.118642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:44.119265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:44.125817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:44.126301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:44.126743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:44.128149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:44.128610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:44.129040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:44.164901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:44.173379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-951d5fad-3e54-4eeb-8b48-828423f4669c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149681 Aug 24 10:35:44.173952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1247/2627] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:44 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:44.257225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:44.265396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dc0af00d-f919-4fcd-9607-f8f458e25b79 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150609 Aug 24 10:35:44.265961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1381/2628] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:44 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:44.770923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:44.771542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:44.777548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:44.778033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:44.778473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:44.779157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:44.779613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:44.780065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:44.911176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:44.914601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:44.920348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-79ec5285-351d-4346-9ab4-1826fa359a49 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153304 Aug 24 10:35:44.920906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1248/2629] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:44 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:45.194867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:45.195480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:45.201818 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:45.202295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:45.202730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:45.203410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:45.204026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:45.204438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:45.288678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:45.289315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:45.295813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:45.296320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:45.296745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:45.297425 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:45.297870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:45.298289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:45.332768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:45.340970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-78f9a442-2325-4bea-9a16-d896116165bc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149971 Aug 24 10:35:45.341542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1382/2630] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:45 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:45.428748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:45.437196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-edcb9d63-ae55-4f06-9502-966f5795e640 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152270 Aug 24 10:35:45.437748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1249/2631] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:45 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:45.942174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:45.942787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:45.948828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:45.949310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:45.949747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:45.950429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:45.950889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:45.951313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:46.095931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:46.104276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5edeb80c-1fec-4f25-9a5f-12d1cdc2bd96 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.166596 Aug 24 10:35:46.104843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1383/2632] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:45 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:46.362797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:46.363374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:46.369729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:46.370212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:46.370646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:46.371324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:46.371774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:46.372250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:46.458405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:46.459118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:46.465725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:46.466216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:46.466689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:46.467385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:46.467848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:46.468298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:46.505823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:46.514121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-44780b9e-8314-4c5f-a3e7-3aab665f106a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.156172 Aug 24 10:35:46.514691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1250/2633] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:46 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:46.595029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:46.603748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a3eeedbe-1972-447e-b6ef-196388ced79a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149546 Aug 24 10:35:46.604348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1384/2634] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:46 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:47.126056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:47.126667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:47.133031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:47.133514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:47.133953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:47.134634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:47.135086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:47.135509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:47.265778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:47.274507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a0e43b68-d4ae-4017-8407-956687d4e226 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152398 Aug 24 10:35:47.275131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1251/2635] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:47 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:47.535843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:47.536435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:47.542859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:47.543343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:47.543782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:47.544504 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:47.544967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:47.545397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:47.625547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:47.626374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:47.633242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:47.633582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:47.634248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:47.634946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:47.635418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:47.635844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:47.675945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:47.684314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-aaf8ee6f-8341-4f2a-9de1-db64b3ea4d1f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152982 Aug 24 10:35:47.684889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1385/2636] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:47 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:47.763690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:47.772655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e825da8e-9f4b-402c-a583-3ab0f0bfaab1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150912 Aug 24 10:35:47.773229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1252/2637] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:47 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:48.295345 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:48.295965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:48.302834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:48.303321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:48.303764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:48.304483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:48.304946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:48.305385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:48.432096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:48.441253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-92d4e62c-0d6c-4678-9546-d3bc5729519c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149168 Aug 24 10:35:48.441854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1386/2638] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:48 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:48.705700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:48.706315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:48.712756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:48.713239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:48.713674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:48.714360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:48.714813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:48.715237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:48.794363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:48.795019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:48.802535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:48.803026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:48.803467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:48.804191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:48.804651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:48.805075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:48.852503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:48.860775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c97fb1f8-b8a1-48c9-a05b-0c15e8f09240 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.159020 Aug 24 10:35:48.861400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1253/2639] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:48 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:48.933343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:48.941482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-663406e4-80e5-401b-823d-8ed1ed93c699 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151014 Aug 24 10:35:48.942048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1387/2640] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:48 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:49.462115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:49.462765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:49.468921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:49.469401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:49.469834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:49.470524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:49.470973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:49.471396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:49.597948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:49.606121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2a28309b-0b75-4f79-bc35-5c58fdb592ee tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147946 Aug 24 10:35:49.606676 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1254/2641] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:49 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:49.883344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:49.884017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:49.890427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:49.890938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:49.891378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:49.892146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:49.892604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:49.893032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:49.963024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:49.963638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:49.969532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:49.970007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:49.970447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:49.971131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:49.971872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:49.971992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:50.023318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:50.026711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:50.032706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b04fc183-e799-43c4-89b9-1e6973fe53f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153769 Aug 24 10:35:50.033462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1388/2642] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:49 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:50.103544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:50.115427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ff6cc27b-548f-4a0d-8ea4-6c8508498a6a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156129 Aug 24 10:35:50.115998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1255/2643] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:49 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:50.627493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:50.628124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:50.634867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:50.635342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:50.635770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:50.636470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:50.636919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:50.637338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:50.762988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:50.771158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e3689ee3-cc84-4184-91c6-2ffae453f3fc tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.147519 Aug 24 10:35:50.771724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1389/2644] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:50 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:51.053973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:51.054585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:51.061026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:51.061509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:51.061944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:51.062622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:51.063074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:51.063522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:51.136304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:51.136931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:51.143138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:51.143751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:51.144037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:51.144832 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:51.145216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:51.145638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:51.203266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:51.211546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2887c274-5cc1-478c-ab23-466a8ca3da26 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.162001 Aug 24 10:35:51.212339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1256/2645] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:51 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:51.278706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:51.287455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1d633a53-5ff0-4fa6-9a0a-b7f83475d297 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154983 Aug 24 10:35:51.288008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1390/2646] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:51 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:51.792589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:51.793211 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:51.799434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:51.799911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:51.800381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:51.801066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:51.801521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:51.801946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:51.929907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:51.937985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e8e16ba6-ca62-4372-bc25-0af5b51bcdd2 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.149590 Aug 24 10:35:51.938548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1257/2647] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:51 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:52.234260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:52.234865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:52.241288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:52.241775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:52.242218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:52.242905 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:52.243369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:52.243799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:52.308682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:52.309305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:52.315624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:52.316175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:52.316759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:52.317457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:52.317912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:52.318343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:52.374221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:52.383157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-541c633c-1029-4e5f-b2d8-17d460f066c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152743 Aug 24 10:35:52.383745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1391/2648] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:52 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:52.455580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:52.464414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-98ec91d8-019a-4c35-87d2-b7ac14f913cc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.159642 Aug 24 10:35:52.464995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1258/2649] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:52 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:52.958727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:52.959340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:52.965240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:52.965726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:52.966229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:52.966910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:52.967362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:52.967794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:53.097787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:53.106499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-193b2bbc-dcb3-4ef6-9740-28aeb398b8a0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151542 Aug 24 10:35:53.107079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1392/2650] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:52 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:53.404198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:53.404763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:53.411724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:53.412238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:53.412675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:53.413355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:53.413813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:53.414237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:53.485410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:53.486038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:53.493248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:53.493717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:53.494151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:53.494835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:53.495284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:53.495705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:53.544374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:53.552601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6a31b6d4-a27d-4a97-ae27-638dddc9d6a2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152376 Aug 24 10:35:53.553164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1259/2651] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:53 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:53.624186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:53.632689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5f291938-7a7c-4588-89c9-c14e57e7a9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151140 Aug 24 10:35:53.633244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1393/2652] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:53 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:54.128142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:54.128668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:54.131036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=78 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:54.136432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:54.136911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:54.137343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:54.138020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:54.138470 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:54.138891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:54.270713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:54.278854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-622be8d5-2082-40c9-beb4-ef6378a11683 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154747 Aug 24 10:35:54.279440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1260/2653] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:54 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:54.574632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:54.575325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:54.581760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:54.582249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:54.582689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:54.583374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:54.583846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:54.584289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:54.654209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:54.654936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:54.662242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:54.662716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:54.663157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:54.663854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:54.664348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:54.664778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:54.712251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:54.720417 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-77e6ac3c-1174-4544-a090-fc97173e15e5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149883 Aug 24 10:35:54.720989 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1394/2654] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:54 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:54.794715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:54.803527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-31961b80-56f1-415f-b071-241b89981af3 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153281 Aug 24 10:35:54.804172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1261/2655] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:54 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:55.300628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:55.301240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:55.303117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=78 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:55.308499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:55.308981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:55.309414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:55.310096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:55.310543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:55.310959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:55.440873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:55.449759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b4cd3dfe-ba54-418a-bd25-208ddddd1464 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.153200 Aug 24 10:35:55.450324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1395/2656] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:55 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:55.742598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:55.743212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:55.749607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:55.750090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:55.750526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:55.751207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:55.751690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:55.752122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:55.825308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:55.826041 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:55.832431 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:55.832916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:55.833351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:55.834036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:55.834487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:55.834908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:55.882749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:55.891058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0572204b-fe42-4ad1-86b6-ba66eadfed55 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152372 Aug 24 10:35:55.891621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1262/2657] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:55 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:55.964943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:55.973191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2c90aff0-e0fc-4eed-84f7-d431ba5d042a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151730 Aug 24 10:35:55.973918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1396/2658] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:55 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:56.470839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:56.471454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:56.477621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:56.478105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:56.478539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:56.479227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:56.479710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:56.480171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:56.609556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:56.618437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-52b6d7ec-8701-43de-9ada-11b1f19022b4 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151535 Aug 24 10:35:56.618998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1263/2659] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:56 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:56.912505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:56.913135 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:56.919621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:56.920132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:56.921125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:56.921808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:56.922262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:56.922682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:57.000483 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:57.001123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:57.007953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:57.008469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:57.008904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:57.009604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:57.010059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:57.010482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:57.058141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:57.066361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c9b7245f-abde-4666-b244-7f8dccb39035 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.158231 Aug 24 10:35:57.067021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1397/2660] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:56 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:57.139099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:57.147288 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a59c2120-e30e-4d0a-8aee-8773d2b4e536 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156037 Aug 24 10:35:57.147850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1264/2661] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:56 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:57.639643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:57.640287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:57.647729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:57.648255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:57.648700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:57.649386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:57.649841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:57.650271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:57.779097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:57.787673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7ab7d530-0a38-4e97-a7e1-7d1b83b5a608 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151945 Aug 24 10:35:57.788264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1398/2662] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:57 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:58.087701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:58.088285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:58.094717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.095200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.095637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.096355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.096805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.097233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.115474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:35:58.168805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:58.169513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:58.175889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.176416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.176858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.178182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.178316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.178671 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.234150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:58.242879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ffa4fca0-f4b8-4c34-b43c-983d9632ef2a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.159570 Aug 24 10:35:58.243550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1265/2663] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:58 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:58.306233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:58.314415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-366b5ef0-98f8-413a-af71-1dd22c21341f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149606 Aug 24 10:35:58.314982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1399/2664] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:58 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:58.761442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:58.761939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:58.767920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.768440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.768875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.769638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.770094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.770529 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.795404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:35:58.808998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:58.809942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:58.816347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.816871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.817311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.818676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.819325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.819759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.874822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-799eb125-4ff9-4ef0-8eea-ab233ed66bec tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "DELETE /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 204 len: 0 microversion: 2.20 time: 0.117287 Aug 24 10:35:58.875691 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1266/2665] 10.222.0.86 () {60 vars in 1280 bytes} [Wed Aug 24 10:35:58 2022] DELETE /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Aug 24 10:35:58.886489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:58.888236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:58.897844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.898383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.898829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.900191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:58.900729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:58.901180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:58.962599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:58.971087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3f5189db-11e0-450a-b19a-c443dcd1eb80 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.165859 Aug 24 10:35:58.971654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1400/2666] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:58 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:59.032774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:59.041528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:59.043040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:59.044241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6d7c8ac4-207a-4ad1-8cb2-d7552bd6180d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1723 microversion: 2.20 time: 0.163521 Aug 24 10:35:59.044785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1267/2667] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:35:58 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1723 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 24 10:35:59.264437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:59.265054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:59.271985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:59.272501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:59.272937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:59.273620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:59.274073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:59.274497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:59.335525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:59.336122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:59.342735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:59.343222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:59.343657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:59.344383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:59.344836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:35:59.345259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:35:59.412285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:59.420585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d55568ea-219e-4eff-bcb4-26d99abd7b33 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.159907 Aug 24 10:35:59.421653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1401/2668] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:59 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:35:59.479404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:35:59.487597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7bc8fa7c-260c-475d-b580-7361b751607b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156642 Aug 24 10:35:59.488204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1268/2669] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:59 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:35:59.992560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:35:59.993179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:35:59.999153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:35:59.999636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:00.000108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:00.000814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:00.001303 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:00.001730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:00.066880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:00.067543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:00.074047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:00.074523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:00.074954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:00.075639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:00.076120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:00.076541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:00.135241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:00.143541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-801bf52b-87e6-4eb9-8533-fa141b388cfa tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.154836 Aug 24 10:36:00.144261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1402/2670] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:35:59 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:00.209723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:00.217802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:00.219280 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:00.220464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-568ab1a3-8c4e-4f3a-8a05-95cbe802b1c7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1723 microversion: 2.20 time: 0.157565 Aug 24 10:36:00.221014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1269/2671] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:36:00 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:36:00.441989 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:00.442598 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:00.449731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:00.450265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:00.450709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:00.451409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:00.451882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:00.452454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:00.510374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:00.511017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:00.517539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:00.518020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:00.518459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:00.519161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:00.519624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:00.520079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:00.651212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:00.659837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7deea944-89fd-467d-b237-7dcfe82769b0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154233 Aug 24 10:36:00.660452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1270/2672] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:00 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:00.686218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:00.694440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5d2d9b93-345e-4c5a-8ac1-2577fe3c6dca tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.256394 Aug 24 10:36:00.695008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1403/2673] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:00 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:01.165234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:01.165840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:01.172334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:01.172818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:01.173249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:01.173932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:01.174381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:01.174795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:01.243048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:01.243887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:01.250328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:01.251065 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:01.252085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:01.252582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:01.253139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:01.253674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:01.325879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:01.336879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-214eb365-6713-4c63-a136-54579e5a2602 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.175505 Aug 24 10:36:01.337728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1271/2674] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:01 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:01.375676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:01.384549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:01.386103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'df9b402589fa4020bc241a4b9c06dd91', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c1ad7a7198d4e78b9d03f00ce65e091', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:01.387293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-30031c8e-6626-456c-8b0b-71f0f48b545d tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 200 len: 1655 microversion: 2.20 time: 0.148314 Aug 24 10:36:01.387853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1404/2675] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:36:01 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 1655 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 24 10:36:01.591049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-730c2abe-4759-4f00-8638-d667aae626d4 req-9a23f957-aae9-43d5-84e1-5c0b0423a44f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9b34f307-a219-4d5f-aa5d-0e3eb79ea936", "name": "network-vif-deleted", "tag": "b7060ba2-b694-4997-b4c4-21985557243b"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:36:01.607063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-730c2abe-4759-4f00-8638-d667aae626d4 req-9a23f957-aae9-43d5-84e1-5c0b0423a44f service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:01.607809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-730c2abe-4759-4f00-8638-d667aae626d4 req-9a23f957-aae9-43d5-84e1-5c0b0423a44f service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:01.608294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-730c2abe-4759-4f00-8638-d667aae626d4 req-9a23f957-aae9-43d5-84e1-5c0b0423a44f service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:01.624355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-730c2abe-4759-4f00-8638-d667aae626d4 req-9a23f957-aae9-43d5-84e1-5c0b0423a44f service nova] Creating event network-vif-deleted:b7060ba2-b694-4997-b4c4-21985557243b for instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 on hv-cinder-85427 Aug 24 10:36:01.636872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-730c2abe-4759-4f00-8638-d667aae626d4 req-9a23f957-aae9-43d5-84e1-5c0b0423a44f service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049022 Aug 24 10:36:01.639508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1272/2676] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:36:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:36:01.682723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:01.683634 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:01.691267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:01.691777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:01.692305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:01.693062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:01.694038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:01.694660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:01.718199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:01.719432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:01.727419 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:01.728431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:01.729150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:01.730157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:01.730857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:01.732197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:01.763424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:01.907064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:01.918554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f7d28783-1b48-402a-9897-e88a84d20922 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.204231 Aug 24 10:36:01.924092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1273/2677] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:01 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:01.924474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:01.932002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-02d1bc1e-501a-451b-b1ee-0d8530319871 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.254252 Aug 24 10:36:01.932738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1405/2678] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:01 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:02.357824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:02.358650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:02.364727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:02.365270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:02.365789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:02.366468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:02.366924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:02.367348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:02.410488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:02.411108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:02.417139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:02.417629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:02.418078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:02.418876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:02.419223 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:02.419651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:02.520304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:02.529891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-df63b351-d5a6-46f5-85ac-e43327d41d40 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.175831 Aug 24 10:36:02.530490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1274/2679] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:02 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:02.593533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] HTTP exception thrown: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Request method failure captured: Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 HTTP/1.1 Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdZQXnG6AIb-u3RcQ-yc69u-eEldWbKWCPZZqv-mHhomvyxd5PLYukdAEAwdUc9jJ1RzIUDFCbcnMnB66M_oSu8G6Lefyyjl8IcxcKnFAn-rEO_qwm4ay-PFgPAx24qS8zgrjL-9XlNGS1kJV6YTlDCxziXPUzeoHs_B9C0DbAn1pE Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Openstack-Nova-Api-Version: 2.20 Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:36:02.595217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: df9b402589fa4020bc241a4b9c06dd91 Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:02.597100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '9b34f307-a219-4d5f-aa5d-0e3eb79ea936'} Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:36:02.601394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:02.603224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:02.604891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Returning 404 to user: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:36:02.604891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-64869690-c811-494b-8f97-40688dee17b2 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 404 len: 111 microversion: 2.20 time: 0.190111 Aug 24 10:36:02.604891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1406/2680] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:36:02 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 111 bytes in 191 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Aug 24 10:36:02.610149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e1816dc6-810b-4742-be36-99a6fae26aad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '.version_select at 0x7fa33d74e820>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:02.630259 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e1816dc6-810b-4742-be36-99a6fae26aad tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-379029465" status: 204 len: 0 microversion: 2.20 time: 0.023848 Aug 24 10:36:02.630834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1275/2681] 10.222.0.86 () {60 vars in 1259 bytes} [Wed Aug 24 10:36:02 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-379029465 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 24 10:36:02.940273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:02.941025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:02.948070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:02.948580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:02.949190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:02.949716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:02.950169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:02.950589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:02.954845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:02.955709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:02.963517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:02.964001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:02.964492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:02.965182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:02.965791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:02.966409 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:03.108560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:03.116919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-db76ca95-eba5-4f60-98df-1d8e712598fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.165982 Aug 24 10:36:03.117495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1276/2682] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:02 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:03.158916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:03.167629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9c9babf8-f449-4350-ad91-3347aeaf8d66 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.231203 Aug 24 10:36:03.168336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1407/2683] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:02 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:03.339001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:03.339488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:03.345368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:03.345840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:03.346266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:03.346935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:03.347379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:03.347798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:03.362317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] HTTP exception thrown: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Request method failure captured: Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 HTTP/1.1 Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdZQXnG6AIb-u3RcQ-yc69u-eEldWbKWCPZZqv-mHhomvyxd5PLYukdAEAwdUc9jJ1RzIUDFCbcnMnB66M_oSu8G6Lefyyjl8IcxcKnFAn-rEO_qwm4ay-PFgPAx24qS8zgrjL-9XlNGS1kJV6YTlDCxziXPUzeoHs_B9C0DbAn1pE Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Openstack-Nova-Api-Version: 2.20 Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:36:03.364024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: df9b402589fa4020bc241a4b9c06dd91 Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.365982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '9b34f307-a219-4d5f-aa5d-0e3eb79ea936'} Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:36:03.366990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.368709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.370435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Returning 404 to user: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:36:03.370435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1cb10bd0-1897-49e5-b757-0dad573526f7 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "DELETE /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 404 len: 111 microversion: 2.20 time: 0.030929 Aug 24 10:36:03.370435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1277/2684] 10.222.0.86 () {60 vars in 1280 bytes} [Wed Aug 24 10:36:03 2022] DELETE /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:36:03.373981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:03.374585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 9b34f307-a219-4d5f-aa5d-0e3eb79ea936] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:03.380379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:03.380855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:03.381313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:03.381990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:03.382464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:03.382882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:03.396171 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] HTTP exception thrown: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Request method failure captured: Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 HTTP/1.1 Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdZQXnG6AIb-u3RcQ-yc69u-eEldWbKWCPZZqv-mHhomvyxd5PLYukdAEAwdUc9jJ1RzIUDFCbcnMnB66M_oSu8G6Lefyyjl8IcxcKnFAn-rEO_qwm4ay-PFgPAx24qS8zgrjL-9XlNGS1kJV6YTlDCxziXPUzeoHs_B9C0DbAn1pE Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Openstack-Nova-Api-Version: 2.20 Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:36:03.398235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: df9b402589fa4020bc241a4b9c06dd91 Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.400098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '9b34f307-a219-4d5f-aa5d-0e3eb79ea936'} Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:36:03.401120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. Aug 24 10:36:03.402758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:36:03.404535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Returning 404 to user: Instance 9b34f307-a219-4d5f-aa5d-0e3eb79ea936 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:36:03.404535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f581e785-7210-4406-99fb-386e8fa6bb09 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936" status: 404 len: 111 microversion: 2.20 time: 0.029422 Aug 24 10:36:03.404535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1408/2685] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:36:03 2022] GET /compute/v2.1/servers/9b34f307-a219-4d5f-aa5d-0e3eb79ea936 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:36:03.550858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:03.551475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:03.557490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:03.557976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:03.558413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:03.559093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:03.559543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:03.559965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:03.688421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:03.698133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d6aca1f0-1bb4-4227-a2fc-8f99e3cdc6f9 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.151093 Aug 24 10:36:03.699097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1278/2686] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:03 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:04.139523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:04.140164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:04.146846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:04.147355 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:04.147796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:04.148535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:04.148994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:04.149803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:04.188923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:04.189486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:04.196335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:04.196829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:04.197267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:04.197978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:04.198767 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:04.199204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:04.287526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:04.296073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9a08ada5-f9fe-4d88-ab4f-625bdce6a0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.160619 Aug 24 10:36:04.296644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1409/2687] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:04 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:04.332913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:04.341113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c07c03d0-effb-4fb4-a18e-882fbb5dbf6c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.156081 Aug 24 10:36:04.341661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1279/2688] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:04 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:04.719662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:04.720304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:04.726705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:04.727183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:04.727617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:04.728324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:04.728779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:04.729204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:04.862232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:04.871224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ba19b14b-67e4-4617-a425-d9d26e982282 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.155374 Aug 24 10:36:04.871781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1410/2689] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:04 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:05.320573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:05.321186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:05.327715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:05.328236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:05.328675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:05.329360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:05.329813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:05.330235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:05.362574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:05.363275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:05.369632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:05.370121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:05.370560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:05.371245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:05.371698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:05.372142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:05.465898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:05.474438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bf8fa3cc-0b63-45c2-8f94-525f9251f567 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.158301 Aug 24 10:36:05.475010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1280/2690] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:05 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:05.513909 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:05.522292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-55856d51-7af8-49fb-af95-de5827cd7601 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.163392 Aug 24 10:36:05.523446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1411/2691] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:05 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:05.892747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:05.893362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:05.899436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:05.899916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:05.900379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:05.901058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:05.901504 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:05.901925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:06.030431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:06.039087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-22783a16-26c0-4d78-971a-7acbcc481a65 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150258 Aug 24 10:36:06.039647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1281/2692] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:05 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:06.496401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:06.497142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:06.503514 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:06.503996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:06.504464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:06.505147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:06.505665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:06.506093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:06.543182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:06.543781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:06.550367 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:06.550854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:06.551287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:06.551978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:06.552459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:06.552901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:06.646061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:06.654255 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d33066a0-8f26-4b58-b7f3-2086dc93ffe6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.161658 Aug 24 10:36:06.654826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1412/2693] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:06 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:06.692100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:06.695562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:06.701004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-44fce3a2-5ac1-40d4-8d6e-33df5ff1a17e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.161849 Aug 24 10:36:06.701561 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1282/2694] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:06 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:07.059822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:07.060455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:07.066337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:07.066819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:07.067260 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:07.067951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:07.068434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:07.068857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:07.209842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:07.218637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-49f215c9-3f59-4ee7-9105-3d0b99ccf908 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.162697 Aug 24 10:36:07.219292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1413/2695] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:07 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:07.676602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:07.677449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:07.685695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:07.686177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:07.686614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:07.687296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:07.687749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:07.688212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:07.721685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:07.722395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:07.729798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:07.730366 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:07.730807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:07.731501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:07.731950 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:07.732398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:07.820123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:07.828726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9198c33a-7d50-4827-8e65-fd283ef2c6bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156389 Aug 24 10:36:07.829387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1283/2696] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:07 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:07.871213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:07.879451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4de074df-e24e-4ca7-9492-641a552a0d8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.161616 Aug 24 10:36:07.880009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1414/2697] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:07 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:08.239636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:08.240365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:08.246300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:08.246780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:08.247214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:08.247895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:08.248370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:08.248797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:08.377105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:08.386060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f9a00613-86de-4196-92d8-461d2104a30c tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.150336 Aug 24 10:36:08.386629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1284/2698] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:08 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:08.850695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:08.851193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:08.857706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:08.858300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:08.858746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:08.859445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:08.859903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:08.860560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:08.901859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:08.902443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:08.928983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:08.930078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:08.930509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:08.931209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:08.931656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:08.936374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:08.964996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:36:09.014301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:09.024106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4d13b62b-d924-4bb6-8237-2add93d73787 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.176598 Aug 24 10:36:09.038037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1415/2699] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:08 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:09.038037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:09.038037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:09.038037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:09.038653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:09.038653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:09.038653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:09.038653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:09.039037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:09.052184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b65e453d-ba49-4108-96a8-b1e1fb4dde06 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "DELETE /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 204 len: 0 microversion: 2.1 time: 0.159906 Aug 24 10:36:09.053027 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1285/2700] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:36:08 2022] DELETE /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 24 10:36:09.063003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:09.063695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:09.070276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:09.070755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:09.071180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:09.071856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:09.072333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:09.072751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:09.174513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:09.184647 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9ef0153a-7a97-4c6d-8597-9a67cad25ce5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.158443 Aug 24 10:36:09.185335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1416/2701] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:09 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:09.208985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:09.218075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-60e1c5b2-7831-4f08-ba88-1da4650cd3f0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1720 microversion: 2.1 time: 0.159670 Aug 24 10:36:09.218642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1286/2702] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:09 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1720 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:09.407621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:09.408227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:09.414221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:09.414712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:09.415149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:09.415842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:09.416320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:09.416749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:09.547637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:09.556289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8351e5c9-d350-4e87-b9fe-6a441bab3736 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.152582 Aug 24 10:36:09.556853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1417/2703] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:09 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:10.045831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:10.046399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:10.052876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:10.053357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:10.053793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:10.054982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:10.055430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:10.055856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:10.193755 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:10.206717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:10.207994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:10.214468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:10.216827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:10.217748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:10.218637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:10.219148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7eaf3233-c5a8-4921-bd24-d4aaec0d5bc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.177230 Aug 24 10:36:10.219304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:10.219726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:10.234296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1287/2704] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:10 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:10.239705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:10.240702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:10.248959 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:10.262997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:10.264375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:10.267015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:10.267581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:10.268073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:10.413451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:10.422145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-028fde33-5966-44df-9ae8-ca0789bb35b2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1720 microversion: 2.1 time: 0.186310 Aug 24 10:36:10.422743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1288/2705] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:10 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1720 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:10.443291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:10.452072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b637981b-d6ad-4542-abab-42efb863e9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.249516 Aug 24 10:36:10.452709 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1418/2706] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:10 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:10.577149 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:10.577764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] [instance: 987654b5-b3b6-434c-a284-d630ab8ab4ad] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:10.583609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:10.584347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:10.584953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:10.585734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:10.586192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:10.586614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:10.722733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bea2aecac254429a823b8765feb6c767', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5294ff8dc07444e5b8758dbae2db7fc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:10.731701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e8d0ba4d-e383-4109-b48d-3aeb1ebd03f1 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad" status: 200 len: 1598 microversion: 2.1 time: 0.158389 Aug 24 10:36:10.732335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1289/2707] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:10 2022] GET /compute/v2.1/servers/987654b5-b3b6-434c-a284-d630ab8ab4ad => generated 1598 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:10.754952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5b78c951-e6cc-484f-b1b3-8d11d6af91e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] Calling method '.version_select at 0x7fa342294160>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:10.756501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-f870eccc-9ed0-42b2-9787-c1932ed7192f req-b149b1e6-d954-4b30-9af5-a5ffedab4de3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5692cc07-3f16-47a4-9993-503b451d9951", "name": "network-vif-deleted", "tag": "c35480cd-41a8-4a27-93df-3ded8cf5eeda"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:36:10.778466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f870eccc-9ed0-42b2-9787-c1932ed7192f req-b149b1e6-d954-4b30-9af5-a5ffedab4de3 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:10.779153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f870eccc-9ed0-42b2-9787-c1932ed7192f req-b149b1e6-d954-4b30-9af5-a5ffedab4de3 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:10.779713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f870eccc-9ed0-42b2-9787-c1932ed7192f req-b149b1e6-d954-4b30-9af5-a5ffedab4de3 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:10.792457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5b78c951-e6cc-484f-b1b3-8d11d6af91e0 tempest-TestMinimumBasicScenario-460256604 tempest-TestMinimumBasicScenario-460256604-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-932602861" status: 202 len: 0 microversion: 2.1 time: 0.043199 Aug 24 10:36:10.793348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1419/2708] 10.222.0.86 () {58 vars in 1269 bytes} [Wed Aug 24 10:36:10 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-932602861 => generated 0 bytes in 45 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:36:10.811540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-f870eccc-9ed0-42b2-9787-c1932ed7192f req-b149b1e6-d954-4b30-9af5-a5ffedab4de3 service nova] Creating event network-vif-deleted:c35480cd-41a8-4a27-93df-3ded8cf5eeda for instance 5692cc07-3f16-47a4-9993-503b451d9951 on hv-cinder-85427 Aug 24 10:36:10.817369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-f870eccc-9ed0-42b2-9787-c1932ed7192f req-b149b1e6-d954-4b30-9af5-a5ffedab4de3 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064074 Aug 24 10:36:10.821402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1290/2709] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:36:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:36:11.241999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:11.242760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:11.250081 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:11.250899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:11.251540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:11.252364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:11.253057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:11.253618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:11.409208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:11.418466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b7cbb5be-09f3-42ee-b41f-46ddcfb2ffcc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.181630 Aug 24 10:36:11.419039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1420/2710] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:11 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:11.446857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:11.447475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:11.453868 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:11.454380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:11.455059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:11.455951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:11.456626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:11.457051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:11.477412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:11.478501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:11.485267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:11.486167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:11.487361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:11.488347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:11.489036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:11.489716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:11.545311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:11.572428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-28acb4ac-22b1-4cd7-b00a-1e3d350afcd2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.131233 Aug 24 10:36:11.573705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1291/2711] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:11 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:11.661044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:11.672199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-64a3fdea-d464-40d2-abfd-7756e446079c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.201262 Aug 24 10:36:11.672930 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1421/2712] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:11 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:12.440795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:12.441728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:12.449103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:12.449759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:12.450007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:12.450731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:12.451178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:12.451596 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:12.599377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:12.601585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:12.601859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:12.611111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ca7ac2f5-ffc9-43be-a5a7-e671e25b1358 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.174234 Aug 24 10:36:12.611725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1292/2713] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:12 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:12.614784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:12.615646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:12.616469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:12.618177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:12.619001 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:12.619834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:12.694812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:12.695703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:12.702117 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:12.702973 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:12.703670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:12.704463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:12.704980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:12.705459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:12.710534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:12.720286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-20f66981-ecc6-40d8-ad10-bf9f440cce4d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.128907 Aug 24 10:36:12.735814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1422/2714] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:12 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:12.854524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:12.862829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-40a306c0-f68e-412f-8623-dd51028478f9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.172657 Aug 24 10:36:12.863391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1293/2715] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:13.633933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:13.634674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:13.641617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:13.642104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:13.642543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:13.643238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:13.643690 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:13.644132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:13.744554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:13.745330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:13.752173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:13.752793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:13.753330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:13.754124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:13.754679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:13.755219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:13.840429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:13.848985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ed8dcddb-f77c-4aeb-ba45-6ee75b2bb004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.109266 Aug 24 10:36:13.849559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1294/2716] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:13 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:13.880517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:13.886723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:13.887339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:13.895126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:13.897152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:13.897850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:13.899079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:13.900129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:13.900786 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:13.906210 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-910205af-fb77-4d1d-951f-ccf2a95fdabd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.276780 Aug 24 10:36:13.907532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1423/2717] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:13 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:14.046951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:14.057811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aa89eb71-fa10-47f8-b277-a7f6aee2bb53 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.176059 Aug 24 10:36:14.058581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1295/2718] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:13 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:14.869962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:14.870581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:14.877593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:14.878089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:14.878530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:14.879238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:14.879693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:14.880150 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:14.899896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:14.928484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:14.929132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:14.936384 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:14.936952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:14.937393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:14.938089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:14.938544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:14.938965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:14.951631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:14.959943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b811a25b-9c50-4699-a195-dd6150901004 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.094295 Aug 24 10:36:14.960532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1424/2719] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:14 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:15.070277 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:15.078787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:15.079160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b55b5f5c-efdf-4d7f-84a0-e02ca659ba09 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154664 Aug 24 10:36:15.079400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:15.079736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1296/2720] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:14 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:15.085602 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:15.086088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:15.087144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:15.087843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:15.088321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:15.088751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:15.215868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:15.224059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-628bc328-101c-4837-8d6e-020edbcf7fc2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149137 Aug 24 10:36:15.224627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1425/2721] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:15 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:15.981893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:15.982507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:15.988909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:15.989388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:15.989823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:15.990508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:15.990962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:15.991381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:16.057306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:16.065430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e75a50e0-9513-47c1-85d8-9fc2241f1c66 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087910 Aug 24 10:36:16.065987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1297/2722] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:15 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:16.101046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:16.101651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:16.108729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:16.109206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:16.109640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:16.110321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:16.110772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:16.111192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:16.238518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:16.245055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:16.245749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:16.246782 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-471831fa-4856-4254-ab00-9b2ec293c1ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149494 Aug 24 10:36:16.247432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1426/2723] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:16 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:16.252095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:16.252617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:16.253054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:16.253748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:16.254215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:16.254638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:16.272628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:16.386339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:16.394716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0db31510-aa77-4d33-a80b-8e9f7f96b65b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153555 Aug 24 10:36:16.395281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1298/2724] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:16 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:17.086426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:17.086952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:17.093070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:17.093554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:17.093987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:17.094669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:17.095120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:17.095543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:17.161036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:17.169794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-681789c6-f510-453f-ad8d-5381c8b495cc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087282 Aug 24 10:36:17.170350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1427/2725] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:17 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:17.268658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:17.269272 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:17.275684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:17.276221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:17.276639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:17.277320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:17.277770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:17.278189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:17.404743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:17.412946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-17bcdf36-2c72-4236-aef2-1f800ef735a6 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148201 Aug 24 10:36:17.413788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1299/2726] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:17 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:17.417096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:17.417718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:17.423744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:17.424281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:17.424748 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:17.425443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:17.425906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:17.426337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:17.554329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:17.562609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9fa78bf6-e4f5-4c19-b3ad-43741407737e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149991 Aug 24 10:36:17.563167 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1428/2727] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:17 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:18.191325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:18.191945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:18.199220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:18.199704 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:18.200176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:18.200867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:18.201320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:18.201742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:18.265748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:18.275243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5e20bb14-3add-47fe-ac8a-f79874d47668 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087207 Aug 24 10:36:18.275852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1300/2728] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:18 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:18.441137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:18.441866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:18.448911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:18.449402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:18.449851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:18.450541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:18.451000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:18.451428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:18.582445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:18.588491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:18.589107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:18.590834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5cb61fc1-15d0-4b07-828a-10708fbfa1fe tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153560 Aug 24 10:36:18.591402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1429/2729] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:18 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:18.595009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:18.595496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:18.595935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:18.596652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:18.597114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:18.597537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:18.729053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:18.737326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7bf9b0a4-94cc-4047-b606-d4adbf7f3867 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152674 Aug 24 10:36:18.737891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1301/2730] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:18 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:19.296147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:19.296754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:19.302643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:19.303123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:19.303558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:19.304271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:19.304720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:19.305137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:19.369851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:19.377973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e7442807-4853-4039-a072-0f4bab0d6bfe tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085755 Aug 24 10:36:19.378521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1430/2731] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:19 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:19.612051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:19.612610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:19.619455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:19.619934 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:19.620388 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:19.621059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:19.621508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:19.621922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:19.749789 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:19.758015 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:19.758538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0cc30a26-4c52-4a05-915a-c2241bc6cb87 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150225 Aug 24 10:36:19.758656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:19.758992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1302/2732] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:19 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:19.765447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:19.765935 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:19.766369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:19.767056 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:19.767506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:19.767931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:19.897487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:19.906116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-61a49de2-4867-40a9-b696-c5b0256f69e9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151837 Aug 24 10:36:19.906664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1431/2733] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:19 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:20.399622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:20.400253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:20.402110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:20.407226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:20.407728 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:20.408189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:20.408863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:20.409305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:20.409718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:20.474898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:20.483396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-eabd62aa-3298-4366-8e03-00826db8b91e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087675 Aug 24 10:36:20.483953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1303/2734] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:20 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:20.780698 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:20.781316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:20.787828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:20.788351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:20.788798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:20.789555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:20.789971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:20.790405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:20.921943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:20.927899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:20.928565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:20.930444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c2a301a0-cbfc-48d2-87c7-3f8263fc149d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153609 Aug 24 10:36:20.931011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1432/2735] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:20 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:20.935592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:20.935765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:20.936250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:20.936805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:20.937263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:20.937695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:21.069850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:21.078096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a026a60b-4f86-4cbf-abfe-1ba5379acf9a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153983 Aug 24 10:36:21.078653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1304/2736] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:20 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:21.504556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:21.505163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:21.511314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:21.511800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:21.512275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:21.513216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:21.513673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:21.514102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:21.578256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:21.586438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-da9510c3-5e4b-4784-a894-d7191caccdfa tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085766 Aug 24 10:36:21.586998 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1433/2737] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:21 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:21.951519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:21.952171 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:21.958647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:21.959128 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:21.959564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:21.960285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:21.961332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:21.961332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:22.088094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:22.096565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d464fb79-2744-4739-b700-5a4f6a864427 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149571 Aug 24 10:36:22.097139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1305/2738] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:21 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:22.098921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:22.099549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:22.105974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:22.106465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:22.106914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:22.107610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:22.108090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:22.108526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:22.238750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:22.247771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-902ad792-3419-4328-920f-2cf5f2ee9313 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152755 Aug 24 10:36:22.248361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1434/2739] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:22 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:22.608580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:22.609222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:22.615445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:22.615930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:22.616394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:22.617077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:22.617537 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:22.617961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:22.683454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:22.691671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-63d02969-fd37-4002-8742-1f6b03a6ceb4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.088136 Aug 24 10:36:22.692273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1306/2740] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:22 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:23.118114 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:23.118714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:23.125299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:23.125794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:23.126238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:23.126937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:23.127416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:23.127860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:23.254986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:23.264560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-57cb6b2f-fb8a-4814-8dac-821446f48a5c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150277 Aug 24 10:36:23.265052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1435/2741] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:23 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:23.269911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:23.270553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:23.278266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:23.278750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:23.279182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:23.279867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:23.280348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:23.280771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:23.409987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:23.418173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a75f5770-5d05-4e9f-b4f6-a259f0fcdc5d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152231 Aug 24 10:36:23.418730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1307/2742] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:23 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:23.712749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:23.713526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:23.719455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:23.719939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:23.720409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:23.721091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:23.721546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:23.721971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:23.789753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:23.798016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b60dd4fa-e531-4e1a-b5cf-6c83eaec3993 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.089099 Aug 24 10:36:23.798576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1436/2743] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:23 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:24.285960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:24.286588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:24.293740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:24.294220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:24.294660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:24.295342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:24.295795 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:24.296258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:24.423845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:24.431992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f81c5b39-eda2-413d-9c23-18672e5a4c94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150004 Aug 24 10:36:24.432599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1308/2744] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:24 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:24.439109 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:24.439750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:24.446638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:24.447131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:24.447613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:24.448370 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:24.448830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:24.449256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:24.577388 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:24.585651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7fe33301-1ebd-430a-9804-68aa6fb72fae tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150816 Aug 24 10:36:24.586216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1437/2745] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:24 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:24.818797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:24.819292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:24.825799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:24.826276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:24.826703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:24.827382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:24.827829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:24.828276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:24.892997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:24.901981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6ede8966-3535-4e08-873b-dfb0d6d4f2d7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087126 Aug 24 10:36:24.902543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1309/2746] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:24 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:25.453444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:25.454057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:25.456408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=87,nova_api:DELETE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:25.461844 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:25.462332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:25.462774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:25.464176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:25.464637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:25.465067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:25.593578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:25.603059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-79271b7a-90af-4f3d-b57d-f8927921b915 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152974 Aug 24 10:36:25.603401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1438/2747] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:25 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:25.608490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:25.609096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:25.615631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:25.616145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:25.616584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:25.617270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:25.617724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:25.618145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:25.747551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:25.755813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a6471419-b308-4c8c-80aa-58d58724d686 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151419 Aug 24 10:36:25.756408 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1310/2748] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:25 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:25.922019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:25.922624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:25.928456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:25.928936 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:25.929376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:25.930049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:25.930500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:25.930925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:25.995530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:26.003559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5b1d3305-37e7-4ec2-bf1c-4baa26187a09 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085480 Aug 24 10:36:26.004136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1439/2749] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:25 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:26.624213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:26.624832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:26.631257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:26.631731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:26.632189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:26.632864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:26.633312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:26.633731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:26.768897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:26.778313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-58107846-fb2e-48c2-8db9-175780b8df83 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.157915 Aug 24 10:36:26.778880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1311/2750] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:26 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:26.779024 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:26.779575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:26.788779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:26.789275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:26.789719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:26.791057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:26.792249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:26.792684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:26.925363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:26.933870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2d67489a-7242-4dfb-9b1b-e43101cf0c6a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.159715 Aug 24 10:36:26.934436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1440/2751] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:26 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:27.024858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:27.025441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:27.032106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:27.032588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:27.033020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:27.033701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:27.034154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:27.034577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:27.100304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:27.108576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c51c40b3-b5ec-4187-a95e-c6cc54533e18 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087615 Aug 24 10:36:27.109136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1312/2752] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:27 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:27.801873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:27.802437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:27.809717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:27.810201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:27.810635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:27.811326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:27.811776 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:27.812247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:27.816430 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:27.954337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:27.955726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:27.956577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:27.962823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:27.963063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-486eef67-12e2-488a-a106-023e001251bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.165583 Aug 24 10:36:27.963301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:27.963642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1441/2753] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:27 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:27.963775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:27.964440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:27.964895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:27.965315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:28.099800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:28.109492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a264f29d-07f3-43ee-bcf8-85947ec75e1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.157739 Aug 24 10:36:28.110074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1313/2754] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:27 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:28.130763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:28.131377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:28.137475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:28.137969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:28.138548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:28.139393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:28.139884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:28.140344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:28.207990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:28.216938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3a056f0c-e46d-487d-af95-3604a9f1155a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.091439 Aug 24 10:36:28.217512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1442/2755] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:28 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:28.985498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:28.986107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:28.992555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:28.993040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:28.993477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:28.994164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:28.994620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:28.995539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:29.126206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:29.130659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:29.131295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:29.134664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c4f14501-190e-4f91-ac36-af0863eefb37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152898 Aug 24 10:36:29.135017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1314/2756] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:28 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:29.138215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:29.139327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:29.139449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:29.140099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:29.140554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:29.140979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:29.238002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:29.238620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:29.244882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:29.245917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:29.246358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:29.247054 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:29.247511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:29.247937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:29.252030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:29.270475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:29.279346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f6f07ce4-4801-4786-95e8-87cd790282be tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152390 Aug 24 10:36:29.279915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1443/2757] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:29 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:29.312542 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:29.320775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4b663a81-0eff-4694-ad93-65498c91784c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086644 Aug 24 10:36:29.321347 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1315/2758] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:29 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:30.156870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:30.157476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:30.163794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:30.164302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:30.164736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:30.165700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:30.165866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:30.166284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:30.297376 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:30.301297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:30.301913 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:30.305624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-211fdc41-04d1-45ba-8397-2c17641c9468 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152602 Aug 24 10:36:30.306189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1444/2759] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:30 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:30.308590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:30.309079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:30.309519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:30.310218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:30.310676 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:30.311106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:30.341838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:30.342541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:30.348580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:30.349161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:30.349669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:30.350428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:30.350952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:30.351385 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:30.418736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:30.427037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-99b47c9b-ab47-4660-b8fa-516c69749948 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.089552 Aug 24 10:36:30.427604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1445/2760] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:30 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:30.443007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:30.451353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ac68c471-5da5-41ad-a23f-bf8485b49a79 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153856 Aug 24 10:36:30.451916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1316/2761] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:30 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:31.326976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:31.327601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:31.334052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:31.334534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:31.334975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:31.335663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:31.336145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:31.336573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:31.448998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:31.450319 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:31.456667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:31.457161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:31.457599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:31.458300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:31.458758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:31.459184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:31.468987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:31.478529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f5ac6779-7487-4f69-b637-e8255d049f8d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155565 Aug 24 10:36:31.479104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1446/2762] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:31 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:31.483562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:31.484208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:31.489983 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:31.490468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:31.490908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:31.491605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:31.492079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:31.492516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:31.522727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:31.532188 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aacdbeb7-95d2-4207-abe7-8688986c48c1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.088469 Aug 24 10:36:31.532970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1317/2763] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:31 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:31.621128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:31.629190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-034dfac4-eead-4ea6-a719-1c52cfa29520 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149154 Aug 24 10:36:31.629744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1447/2764] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:31 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:32.121360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:36:32.121581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:36:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:36:32.499788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:32.500434 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:32.506894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:32.507401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:32.507841 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:32.508558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:32.509015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:32.509477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:32.553178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:32.553795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:32.561050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:32.561546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:32.561987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:32.562722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:32.563183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:32.563613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:32.629058 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:32.637928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d7049733-ea32-4786-a32a-00372335e83e tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.088609 Aug 24 10:36:32.638498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1448/2765] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:32 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:32.642439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:32.650338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:32.650826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-eab3a61a-541c-469e-99e3-9bb8a1268cb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154897 Aug 24 10:36:32.651027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:32.651399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1318/2766] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:32 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:32.656884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:32.657485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:32.657940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:32.658635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:32.659082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:32.659503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:32.803096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:32.811814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d368a8e2-2bec-4f12-84dd-b9ce50a607cc tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.165197 Aug 24 10:36:32.812399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1449/2767] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:32 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:33.660005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:33.660621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:33.667513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:33.669013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:33.669157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:33.669797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:33.670236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:33.670656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:33.673525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:33.674220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:33.680639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:33.681216 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:33.681725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:33.682481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:33.683036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:33.683467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:33.733056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:33.741338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f4ccb602-19c5-4b07-a7df-0e28e52e8199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086126 Aug 24 10:36:33.741906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1319/2768] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:33 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:33.814074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:33.822225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-95453922-6d6b-4f49-a2a7-aa6ba8f93a37 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152851 Aug 24 10:36:33.822780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1450/2769] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:33 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:33.833185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:33.833798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:33.840605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:33.841084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:33.841522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:33.842201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:33.842652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:33.843074 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:33.972869 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:33.981327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a024929d-7f42-4c81-8f86-c9ad5c5695d7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151939 Aug 24 10:36:33.981895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1320/2770] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:33 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:34.762783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:34.763402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:34.770197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:34.770685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:34.771128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:34.771825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:34.772314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:34.772746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:34.838107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:34.843378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:34.843999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:34.846937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1b14da4b-b5da-4065-9c85-94a9fbb12420 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.088466 Aug 24 10:36:34.847708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1451/2771] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:34 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:34.851825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:34.852461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:34.853338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:34.853643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:34.854091 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:34.854511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:34.991014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:35.002942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:35.003568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:35.003980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fd649a64-bb4f-413e-b727-150a9a0032ba tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.164103 Aug 24 10:36:35.004608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1321/2772] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:34 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:35.009760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:35.010268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:35.010714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:35.011418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:35.012477 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:35.012912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:35.142106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:35.150365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0ea55bf9-252c-47d8-9fe6-8b3951261898 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151332 Aug 24 10:36:35.150924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1452/2773] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:34 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:35.868196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:35.868843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:35.875812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:35.876338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:35.876778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:35.877467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:35.877924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:35.878350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:35.940336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:35.952082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c7838071-4ebc-4034-917d-62c0339b899d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087763 Aug 24 10:36:35.952650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1322/2774] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:35 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:36.026104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:36.026699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:36.033046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:36.033535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:36.033978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:36.034665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:36.035119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:36.035554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:36.162298 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:36.171017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:36.171254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-63193bdb-0825-4f79-b1be-0ea42caa20f7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149133 Aug 24 10:36:36.171613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:36.171861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1453/2775] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:36 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:36.177453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:36.177926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:36.178358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:36.179039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:36.179489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:36.179906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:36.306471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:36.314635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cf42e3c9-7ed8-444e-9b57-db48b66991c2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.147864 Aug 24 10:36:36.315197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1323/2776] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:36 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:36.972614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:36.973220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:36.979021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:36.979502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:36.979939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:36.980715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:36.981169 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:36.981588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:37.047862 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:37.056246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e79d842c-6a42-4387-8777-8f4aa4fc9104 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087401 Aug 24 10:36:37.056945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1454/2777] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:36 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:37.192977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:37.193582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:37.200756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:37.201231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:37.201660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:37.202335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:37.202779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:37.203198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:37.334754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:37.335361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:37.337050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:37.341991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:37.342480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:37.342916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:37.343605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:37.344095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:37.344501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:37.345564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-18899f9a-11e4-4057-b842-3fad01a2f494 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156357 Aug 24 10:36:37.346128 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1324/2778] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:37 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:37.472128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:37.480478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fbfc0209-4000-4850-ae77-cd3cf575b34e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149534 Aug 24 10:36:37.481105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1455/2779] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:37 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:38.077761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:38.078279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:38.084260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:38.084739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:38.085169 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:38.085848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:38.086298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:38.086720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:38.159001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:38.167891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-87a3664d-8e56-4982-a66d-add1d620479c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.094083 Aug 24 10:36:38.169124 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1325/2780] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:38 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:38.366854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:38.367428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:38.373805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:38.374288 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:38.374725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:38.375412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:38.375865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:38.376316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:38.504100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:38.505020 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:38.506704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:38.514002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:38.514493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:38.514929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:38.515622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:38.516238 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c33cd848-51a2-4a15-bde5-27436b29d38d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153007 Aug 24 10:36:38.516435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:38.516816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1456/2781] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:38 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:38.517037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:38.653183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:38.661426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e0a9ffa7-3b29-4b7f-8f0f-34687c3e73e3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.163748 Aug 24 10:36:38.661993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1326/2782] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:38 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:39.189540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:39.190248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:39.196251 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:39.196740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:39.197186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:39.197880 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:39.198360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:39.198777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:39.263063 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:39.266725 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:39.272681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b771792f-32f4-45bb-be5a-c4a69440509d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087042 Aug 24 10:36:39.273245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1457/2783] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:39 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:39.537721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:39.538332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:39.545539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:39.546148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:39.546589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:39.547278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:39.547734 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:39.548190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:39.680410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:39.682866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:39.683474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:39.688613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d01973da-5a3e-4cd3-b3c2-b56655a6732b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154712 Aug 24 10:36:39.689732 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1327/2784] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:39 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:39.690787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:39.691289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:39.691736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:39.692557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:39.693168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:39.693606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:39.904712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:39.913028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0426efe1-24bc-44c9-9486-1b116a7d6fe2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.234112 Aug 24 10:36:39.913675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1458/2785] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:39 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:40.294326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:40.294947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:40.300904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:40.301387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:40.301822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:40.302503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:40.302954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:40.303373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:40.364334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:40.372495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-284beba5-1699-492d-8348-8c4fcb02a49c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082782 Aug 24 10:36:40.373048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1328/2786] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:40 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:40.710306 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:40.710946 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:40.718008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:40.718484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:40.718917 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:40.719595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:40.720088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:40.720509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:40.852931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:40.861190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e7c0b2e9-92f2-48d4-95ff-fd234ac14a33 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154699 Aug 24 10:36:40.861756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1459/2787] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:40 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:40.935366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:40.936061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:40.942595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:40.943078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:40.943514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:40.944233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:40.944687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:40.945110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:41.077840 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:41.086045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f673f00a-ee87-45cb-b1ef-d939043225f2 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.155587 Aug 24 10:36:41.086607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1329/2788] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:40 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:41.393822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:41.394360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:41.400343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:41.400829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:41.401281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:41.401978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:41.402434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:41.402870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:41.463019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:41.471226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6d939bd5-4539-4827-a9b8-c3c9666f5093 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.081988 Aug 24 10:36:41.471783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1460/2789] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:41 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:41.883477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:41.884118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:41.890068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:41.890545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:41.890978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:41.891923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:41.892401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:41.892824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:42.024489 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:42.033469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6f7939b4-f7b6-43c0-a3ba-1f2f93b6c7ff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153821 Aug 24 10:36:42.034013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1330/2790] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:41 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:42.106784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:42.107398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:42.113965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:42.114453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:42.114895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:42.115582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:42.116061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:42.116494 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:42.254704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:42.264491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-49d4c349-6426-4c13-b4c0-652fe1c220b4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.161539 Aug 24 10:36:42.265054 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1461/2791] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:42 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:42.493737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:42.494349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:42.500817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:42.501301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:42.501735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:42.502418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:42.502874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:42.503301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:42.565354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:42.568775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:42.574554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c08d575e-1b77-4e31-a338-238e78776e96 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084711 Aug 24 10:36:42.575771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1331/2792] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:42 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:43.054649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:43.055184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:43.061826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:43.062321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:43.062757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:43.063446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:43.063900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:43.064365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:43.191999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:43.200117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d57bb047-bdea-4bf9-8d48-664aa86e1f94 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149423 Aug 24 10:36:43.201271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1462/2793] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:43 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:43.285644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:43.286289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:43.292904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:43.293387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:43.293825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:43.294595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:43.295053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:43.295478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:43.424929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:43.433121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7a21852e-0d1e-4273-90d3-41c9d843b9ee tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151338 Aug 24 10:36:43.433686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1332/2794] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:43 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:43.595368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:43.595924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:43.601872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:43.602364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:43.602806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:43.603528 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:43.603988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:43.604448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:43.666508 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:43.675763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-90cf1daa-a44f-41f6-a06c-6ea47f488e65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084279 Aug 24 10:36:43.676357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1463/2795] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:43 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:44.222078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:44.222689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:44.228675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:44.229155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:44.229588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:44.230265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:44.230715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:44.231136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:44.360297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:44.368373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5ad58cea-1ca2-40e3-938c-18ae9be0f8bc tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150170 Aug 24 10:36:44.368935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1333/2796] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:44 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:44.454929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:44.455543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:44.461954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:44.462439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:44.462904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:44.463615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:44.464092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:44.464520 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:44.595335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:44.603476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cc2c2b80-ba9c-4b23-9b57-d4a6c79e5cf0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152875 Aug 24 10:36:44.604027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1464/2797] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:44 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:44.697099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:44.697705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:44.704282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:44.704757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:44.705181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:44.705847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:44.706294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:44.706708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:44.767069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:44.775447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ece27d87-5f37-47a0-90ff-897034249a7a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082190 Aug 24 10:36:44.776016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1334/2798] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:44 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:45.388798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:45.389412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:45.395799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:45.396317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:45.396759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:45.397450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:45.397903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:45.398328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:45.527435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:45.536097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-95f4e7e3-5be1-4679-9941-681195936a31 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151161 Aug 24 10:36:45.536649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1465/2799] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:45 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:45.624526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:45.625218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:45.631731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:45.632266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:45.632709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:45.633393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:45.633850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:45.634276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:45.765692 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:45.774262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b676f7d1-3df9-461a-9a15-59ee2af2323f tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153609 Aug 24 10:36:45.774826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1335/2800] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:45 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:45.796624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:45.797235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:45.803085 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:45.803559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:45.803991 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:45.804686 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:45.805136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:45.805554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:45.865722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:45.873766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3631d352-8f87-4a35-820e-0f38ff1af7fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.080924 Aug 24 10:36:45.874317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1466/2801] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:45 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:46.557687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:46.558338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:46.565168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:46.565655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:46.566089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:46.566775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:46.567226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:46.567656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:46.698165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:46.706408 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3f429413-845a-4f73-b115-3aea77c850eb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152662 Aug 24 10:36:46.706976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1336/2802] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:46 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:46.795837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:46.796633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:46.803262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:46.803751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:46.804220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:46.804911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:46.805369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:46.805797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:46.894680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:46.895318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:46.901273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:46.901766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:46.902851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:46.903029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:46.903479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:46.904445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:46.936801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:46.945338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-97c24027-fac8-43af-8ea9-1becf90fbf85 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153848 Aug 24 10:36:46.946048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1467/2803] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:46 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:46.966036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:46.974750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e236a661-403a-4bc1-b321-e531d5559051 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084357 Aug 24 10:36:46.975310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1337/2804] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:46 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:47.726876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:47.727502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:47.733563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:47.734048 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:47.734493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:47.735182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:47.735648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:47.736101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:47.865532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:47.874199 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9f7cf38b-2892-44de-8cd4-ef98177414f2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151177 Aug 24 10:36:47.874760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1468/2805] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:47 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:47.966678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:47.967291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:47.973838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:47.974322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:47.974797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:47.975482 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:47.975939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:47.976416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:47.996269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:47.996878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:48.003398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:48.003877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:48.004372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:48.005055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:48.005503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:48.005922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:48.070129 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:48.078622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-798bcb85-2b76-4a61-b8c5-64e8708c33db tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086264 Aug 24 10:36:48.079181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1469/2806] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:47 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:48.111793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:48.120132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-335d3b2c-2ed1-4d0c-a0c2-faca44b25d0b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.157289 Aug 24 10:36:48.120700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1338/2807] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:47 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:48.896036 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:48.896673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:48.903077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:48.903592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:48.904031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:48.905305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:48.905757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:48.906184 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:49.035092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:49.043474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d72a87cf-b467-48c2-bcd3-43c4deacd2b5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151373 Aug 24 10:36:49.043794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1470/2808] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:48 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:49.098669 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:49.099260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:49.106103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:49.106103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:49.106543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:49.107220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:49.107671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:49.108120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:49.143162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:49.143781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:49.149775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:49.150287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:49.150770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:49.151471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:49.151929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:49.152392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:49.169452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:49.177687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c6a09383-e1ba-476c-b240-1bfd9d8acf19 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082843 Aug 24 10:36:49.178257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1339/2809] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:49 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:49.283110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:49.291075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-be9db80e-89bc-4bbe-936b-9c39e4e9aa8a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152635 Aug 24 10:36:49.291633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1471/2810] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:49 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:50.065566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:50.077115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:50.077115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:50.077115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:50.077492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:50.078071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:50.078525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:50.078954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:50.198890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:50.199518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:50.205938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:50.206427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:50.206851 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:50.207336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:50.207545 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:50.208002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:50.208453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:50.215785 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6718dd22-803f-4f45-b296-c22838290e80 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154341 Aug 24 10:36:50.215995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1340/2811] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:50 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:50.272570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:50.280884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-49297784-5e5a-4bef-932a-57bb0126a4ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085993 Aug 24 10:36:50.281433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1472/2812] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:50 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:50.311791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:50.312439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:50.318474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:50.318901 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:50.319339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:50.320022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:50.320503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:50.320929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:50.451128 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:50.459387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-09776d6b-0829-4e53-88d2-a31fe57191ec tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152118 Aug 24 10:36:50.459977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1341/2813] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:50 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:51.236451 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:51.237075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:51.243649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:51.244168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:51.244615 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:51.245309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:51.245767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:51.246228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:51.300627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:51.301253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:51.303583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=78 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:51.309140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:51.309630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:51.310099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:51.311295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:51.311750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:51.312221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:51.376109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:51.380962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:51.384372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-103067c1-6445-41cf-9407-7c93a06aa518 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087479 Aug 24 10:36:51.384940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1342/2814] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:51 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:51.389689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-13df7374-e29e-4abd-bf79-eec2a2e0e54a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156501 Aug 24 10:36:51.390233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1473/2815] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:51 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:51.480196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:51.480807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:51.486856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:51.487287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:51.487727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:51.488438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:51.488897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:51.489325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:51.617257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:51.625771 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-59146657-0b81-4170-8356-6a8489895611 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149790 Aug 24 10:36:51.626329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1343/2816] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:51 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:52.406047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:52.406674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:52.410563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:52.411172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:52.413274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:52.413769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:52.414218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:52.414910 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:52.415369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:52.415797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:52.417478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:52.417946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:52.418374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:52.419047 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:52.419495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:52.419912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:52.434151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:52.479712 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:52.489061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-87a0c993-f19e-4d04-8ead-4b03d6fefbbf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086335 Aug 24 10:36:52.489061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1474/2817] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:52 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:52.549791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:52.557939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9b360d5b-ccbe-4867-90cb-5f085a903f17 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151263 Aug 24 10:36:52.558558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1344/2818] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:52 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:52.647391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:52.647995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:52.653867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:52.654343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:52.654778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:52.655453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:52.655903 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:52.656419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:52.783799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:52.792097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-09008453-56d4-4424-8d16-9637d882d28a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148739 Aug 24 10:36:52.792657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1475/2819] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:52 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:53.509899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:53.510510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:53.517057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:53.517544 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:53.517980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:53.518921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:53.519374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:53.519800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:53.579685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:53.580334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:53.580546 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:53.586867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:53.587287 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:53.587729 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:53.588454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:53.588575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-243c1426-536e-492b-b9ef-1aa89f8047da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082639 Aug 24 10:36:53.588912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:53.589122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1345/2820] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:53.589340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:53.721453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:53.729696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3bb19378-39ce-4020-8c7f-a29b478c1fe1 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153889 Aug 24 10:36:53.730259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1476/2821] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:53 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:36:53.814645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:53.815251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:53.821170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:53.821648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:53.822080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:53.822757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:53.823205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:53.823630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:53.955401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:53.963492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f0c42491-6792-4e6b-aff0-a2d99da24361 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153287 Aug 24 10:36:53.964097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1346/2822] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:53 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:54.610963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:54.611526 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:54.617819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:54.618294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:54.618726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:54.619404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:54.619854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:54.620304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:54.682270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:54.690441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c5e22cf6-4bc8-4445-8160-7a6792e40188 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084108 Aug 24 10:36:54.690996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1477/2823] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:54 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:54.750640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:54.751251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:54.757707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:54.758185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:54.758616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:54.759301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:54.759750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:54.760205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:54.892356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:54.900720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2d525ff5-23e9-4330-bf74-ae15eac04d39 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153965 Aug 24 10:36:54.901293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1347/2824] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:54 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:54.984398 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:54.985016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:54.990887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:54.991369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:54.991806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:54.992517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:54.992992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:54.993400 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:55.119112 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:55.127816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0f9bb257-c951-490f-8481-3505611ee727 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.147672 Aug 24 10:36:55.128404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1478/2825] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:54 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:55.711173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:55.711816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:55.717875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:55.718353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:55.718788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:55.719462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:55.719912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:55.720361 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:55.738340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:55.783158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:55.791496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-765dc819-ab2a-46a4-98fd-8211e80e2847 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084088 Aug 24 10:36:55.791995 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1348/2826] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:55 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:55.923310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:55.923964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:55.926299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=82 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:36:55.931643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:55.932149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:55.932586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:55.933268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:55.933720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:55.934141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:56.063911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:56.072116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c10f23be-a59c-415c-8941-d2eefec3e85a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153304 Aug 24 10:36:56.072687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1479/2827] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:55 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:56.149735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:56.150345 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:56.156758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:56.157239 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:56.157670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:56.158383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:56.158827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:56.159245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:56.288678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:56.296768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-16b4e9d9-ba79-47f0-b893-27af22b85c03 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150991 Aug 24 10:36:56.297339 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1349/2828] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:56 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:56.814461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:56.815068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:56.820939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:56.821415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:56.821855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:56.822525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:56.822973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:56.823392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:56.884138 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:56.892246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-42c38905-6ecf-4783-badd-e40c7e33c13a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.081617 Aug 24 10:36:56.892805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1480/2829] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:56 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:57.093898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:57.094508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:57.100947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:57.101428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:57.101870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:57.102556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:57.103010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:57.103433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:57.230173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:57.238325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d66e5936-8395-407e-bb0e-2133d8023e08 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148980 Aug 24 10:36:57.238893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1350/2830] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:57 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:57.317672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:57.318284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:57.324704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:57.325179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:57.325614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:57.326295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:57.326750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:57.327219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:57.452483 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:57.461532 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b7d21f9d-7da0-40cc-a870-84ddc4a83aff tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.147611 Aug 24 10:36:57.462101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1481/2831] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:57 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:57.913556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:57.914166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:57.920068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:57.920563 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:57.921528 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:57.922207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:57.922659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:57.923081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:57.982139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:57.990772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-02125dab-bde5-45db-a23d-96d1c4eb1712 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.081086 Aug 24 10:36:57.991333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1351/2832] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:57 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:58.259416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:58.260028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:58.266123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:58.266620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:58.267059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:58.267757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:58.268248 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:58.268678 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:58.400646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:58.409985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-28f0fc56-c881-4010-95fb-ddcddefdf7d2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154450 Aug 24 10:36:58.410558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1482/2833] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:58 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:58.483304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:58.483962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:58.490374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:58.490854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:58.491281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:58.491954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:58.492431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:58.492849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:58.621133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:58.629287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7293e725-78a9-479c-a8df-1960fb5dce89 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149903 Aug 24 10:36:58.629843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1352/2834] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:58 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:59.012080 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:59.012694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:59.019480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:59.019959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:59.020419 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:59.021099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:59.021550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:59.021970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:59.082563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:59.090613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4ff75144-3f8f-4909-ba57-d0dc82f620df tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082403 Aug 24 10:36:59.091190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1483/2835] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:59 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:59.432292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:59.432908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:59.439515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:59.439996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:59.440452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:59.441137 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:59.441587 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:59.442012 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:59.573522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:59.582395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3a67fa24-ade9-4e34-80d6-db2c89c125c4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153992 Aug 24 10:36:59.582953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1353/2836] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:59 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:36:59.650478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:36:59.651098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:36:59.657677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:59.658170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:59.658611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:59.659299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:36:59.659759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:36:59.660225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:36:59.788140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:36:59.796670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-be06a911-2910-44a4-9e8f-1003f45efbe7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149803 Aug 24 10:36:59.796920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1484/2837] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:36:59 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:00.112987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:00.113539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:00.119447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:00.119925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:00.120389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:00.121061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:00.121503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:00.121916 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:00.182490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:00.191844 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e09d0ffb-dfe4-4cfd-8755-e33b9f1081e3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083438 Aug 24 10:37:00.192665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1354/2838] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:00 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:00.603498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:00.604151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:00.610808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:00.611294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:00.611736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:00.612463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:00.612924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:00.613355 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:00.742566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:00.750735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-91b94e47-fb11-4676-9adf-cadc9979f923 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151146 Aug 24 10:37:00.751291 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1485/2839] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:00 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:00.817058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:00.817698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:00.824206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:00.824694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:00.825135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:00.825826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:00.826283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:00.826711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:00.956475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:00.964649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ad9ff6cf-8782-44b0-8d30-c00404c842a8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151455 Aug 24 10:37:00.965206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1355/2840] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:00 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:01.213583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:01.214205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:01.220151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:01.220645 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:01.221091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:01.221781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:01.222236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:01.222674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:01.285149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:01.293312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-706031d4-57c8-4b30-a0ba-983e10e4dff3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083680 Aug 24 10:37:01.293878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1486/2841] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:01 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:01.772403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:01.772956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:01.779284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:01.779773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:01.780237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:01.781556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:01.782009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:01.782436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:01.908421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:01.916621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d5fff4aa-4512-42e0-af07-be4021e770a5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148437 Aug 24 10:37:01.917181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1356/2842] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:01 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:01.985541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:01.986158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:01.992761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:01.993250 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:01.993689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:01.994387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:01.994849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:01.995273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:02.126228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:02.134444 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a53700c3-5965-400e-9077-cda7324dc034 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152804 Aug 24 10:37:02.135004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1487/2843] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:01 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:02.314613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:02.315251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:02.321138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:02.321617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:02.322045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:02.322720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:02.323165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:02.323582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:02.386294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:02.394316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d20d5a52-6ffb-4622-9a78-f0dfc6dff15b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084195 Aug 24 10:37:02.395164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1357/2844] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:02 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:02.937803 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:02.938423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:02.944957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:02.945449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:02.945892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:02.946583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:02.947046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:02.947475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:03.072281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:03.081112 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b8a8faf3-10b3-40cb-a9da-b70f728b8947 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.147193 Aug 24 10:37:03.081999 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1488/2845] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:02 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:03.156725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:03.157365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:03.163680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:03.164186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:03.164623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:03.165296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:03.165737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:03.166152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:03.416465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:03.417262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:03.423970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:03.424495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:03.424934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:03.425629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:03.426100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:03.426513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:03.489474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:03.497821 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ec3c8192-373c-4fb6-976a-9996f9738d90 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086111 Aug 24 10:37:03.498463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1489/2846] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:03 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:03.536371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:03.544879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7ed2f52c-6d42-4005-a8fe-c260367aa21e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.392323 Aug 24 10:37:03.545207 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1358/2847] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:03 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:04.102848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:04.103488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:04.110793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:04.111283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:04.111723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:04.112445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:04.112901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:04.113330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:04.245163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:04.253241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cf1cfcd4-fb8c-48b1-ba3c-a35540f659af tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154352 Aug 24 10:37:04.253801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1490/2848] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:04 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:04.519245 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:04.519828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:04.526443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:04.526943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:04.527368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:04.528059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:04.528505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:04.528930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:04.564932 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:04.565548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:04.571719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:04.572266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:04.572739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:04.573440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:04.573926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:04.574371 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:04.579112 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:04.590175 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:04.599885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2ae3b403-3d0a-4b30-846c-c215e18ac5d9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084840 Aug 24 10:37:04.600538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1359/2849] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:04 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:04.705517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:04.713673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-651fb881-5a7d-48a5-8e3c-cd465d6f3fe6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152683 Aug 24 10:37:04.714240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1491/2850] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:04 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:05.275177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:05.275827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:05.282240 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:05.282721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:05.283161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:05.283877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:05.284371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:05.284799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:05.414736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:05.423792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-26fb2f1a-dce7-44f0-bb9a-53c7b09ec37b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152468 Aug 24 10:37:05.424381 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1360/2851] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:05 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:05.621032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:05.621641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:05.627836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:05.628349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:05.628785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:05.629466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:05.629931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:05.630350 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:05.691655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:05.699794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7098e18a-3d71-462f-ba63-599cb8c726ea tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082613 Aug 24 10:37:05.700384 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1492/2852] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:05 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:05.735677 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:05.736344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:05.742689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:05.743177 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:05.743656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:05.744364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:05.744820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:05.745251 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:05.873601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:05.881896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-54b000ce-c944-485a-9f23-326a3a066b81 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150753 Aug 24 10:37:05.882466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1361/2853] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:05 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:06.444848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:06.445470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:06.451319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:06.451799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:06.452537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:06.453033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:06.454181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:06.454610 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:06.584591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:06.592701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-045e12d0-336a-4ca2-85c8-62bc449a424c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151771 Aug 24 10:37:06.593258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1493/2854] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:06 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:06.720496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:06.721112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:06.727131 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:06.727612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:06.728066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:06.728756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:06.729212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:06.729637 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:06.791665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:06.800654 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-77a9d98a-5003-4543-96f2-916418882e92 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083976 Aug 24 10:37:06.801215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1362/2855] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:06 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:06.902201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:06.902807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:06.909253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:06.909734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:06.910165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:06.910843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:06.911293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:06.911713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:07.040561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:07.049713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6ed63c9e-07c9-4fa9-98b6-d1278a7516fe tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151339 Aug 24 10:37:07.050283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1494/2856] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:06 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:07.614421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:07.615065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:07.621013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:07.621494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:07.621931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:07.622612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:07.623072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:07.623659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:07.752002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:07.760165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5dfa19c9-7731-45e6-a144-5a75c260411d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150161 Aug 24 10:37:07.760736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1363/2857] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:07 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:07.885959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:07.886578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:07.893134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:07.893616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:07.894052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:07.894738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:07.895193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:07.895908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:07.956445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:07.964591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2ccf4678-cb75-4dec-8b6a-32128a43c3f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.146479 Aug 24 10:37:07.965146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1495/2858] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:07 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:08.133827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:08.134435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:08.140314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:08.140788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:08.141219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:08.141890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:08.142335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:08.142748 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:08.267650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:08.276132 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-74b3c719-6f93-40ef-a454-915c3bf57441 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.209275 Aug 24 10:37:08.276647 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1364/2859] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:08.781379 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:08.781992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:08.788035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:08.788542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:08.788982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:08.789664 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:08.790117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:08.790541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:08.919304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:08.928329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-177910d6-3d17-4e98-8789-d20e6f67cd85 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150552 Aug 24 10:37:08.928669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1496/2860] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:08 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:08.986356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:08.986999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:08.993479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:08.993964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:08.994405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:08.995086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:08.995538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:08.995965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:08.999907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:09.057253 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:09.065592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-10654231-a3e0-4c38-831f-3cb67a6d8649 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083593 Aug 24 10:37:09.066163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1365/2861] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:08 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:09.298205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:09.298829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:09.304863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:09.305296 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:09.305735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:09.306421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:09.306877 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:09.307301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:09.431846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:09.439933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5c9fa5ab-aa05-4fa1-98e8-6fc6f301c50a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.146219 Aug 24 10:37:09.440509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1497/2862] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:09 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:09.950150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:09.950765 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:09.957411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:09.957894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:09.958355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:09.959041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:09.959491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:09.959939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:10.086947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:10.087575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:10.091138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:10.094443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:10.094938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:10.095382 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:10.096094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:10.096552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:10.096981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:10.099560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-854ba704-ec82-4f2a-a9d9-26bd76d0d7c2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153336 Aug 24 10:37:10.100389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1366/2863] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:09 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:10.162801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:10.171157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d194ea1c-caa3-4930-b216-af0b5810629c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.088188 Aug 24 10:37:10.171727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1498/2864] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:10 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:10.460543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:10.461180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:10.467261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:10.467750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:10.468214 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:10.468905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:10.469358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:10.469914 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:10.596948 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:10.604985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d2eae351-ee0e-453f-983c-ee65eaabb5d5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148330 Aug 24 10:37:10.605550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1367/2865] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:10 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:11.120313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:11.120892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:11.127275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:11.127759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:11.128224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:11.128908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:11.129364 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:11.129798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:11.192066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:11.192774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:11.199070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:11.199917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:11.200383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:11.201071 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:11.201523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:11.201950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:11.267517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:11.272794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:11.276578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d23d00c6-560b-4de6-a122-52fa1d81b66a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.160113 Aug 24 10:37:11.277164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1499/2866] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:11 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:11.281284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-76e789f4-61ee-4a4f-abb0-04a689641a87 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.093244 Aug 24 10:37:11.281852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1368/2867] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:11 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:11.631621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:11.632757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:11.643396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:11.644318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:11.645060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:11.646434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:11.647019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:11.647737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:11.804476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:11.812600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dec3110d-2095-43c0-a3ff-93f64837618e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.188955 Aug 24 10:37:11.813165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1500/2868] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:11 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:12.299358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:12.300134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:12.304739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:12.305397 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:12.306605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:12.307090 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:12.307515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:12.308230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:12.308681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:12.309099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:12.311669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:12.312189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:12.312642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:12.313331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:12.313911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:12.314347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:12.379925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:12.388233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ee2dea07-b040-4616-ba96-951f03967484 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087270 Aug 24 10:37:12.388804 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1501/2869] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:12 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:12.437817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:12.445905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-085f9186-aa90-40a4-8f64-47cf0f319c21 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151075 Aug 24 10:37:12.446457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1369/2870] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:12 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:12.834553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:12.835168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:12.841053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:12.841536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:12.841979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:12.842663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:12.843121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:12.843546 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:12.968621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:12.977377 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d7b619ff-6df4-4005-a042-df20b3380bb9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.146656 Aug 24 10:37:12.977934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1502/2871] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:13.408867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:13.409433 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:13.415780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:13.416299 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:13.416739 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:13.417427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:13.417882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:13.418322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:13.466724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:13.467321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:13.473727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:13.474219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:13.474667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:13.475599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:13.476078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:13.476516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:13.482496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:13.491120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-60287d2a-ec4b-45a7-b354-48142eac89e2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086289 Aug 24 10:37:13.491679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1370/2872] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:13 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:13.603117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:13.611849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c9bd67ef-1571-4244-97fd-62bafde84806 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149001 Aug 24 10:37:13.612443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1503/2873] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:13 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:13.998614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:13.999226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:14.005232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:14.005715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:14.006154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:14.006839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:14.007293 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:14.007719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:14.137650 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:14.145897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3881becf-d9f2-414a-9ad0-f956675641ab tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151174 Aug 24 10:37:14.146711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1371/2874] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:13 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:14.512292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:14.512888 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:14.519754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:14.520272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:14.520714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:14.521395 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:14.522115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:14.522538 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:14.586771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:14.595661 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-387faab9-30c4-44a9-a1d0-cacc30e96d7c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.087308 Aug 24 10:37:14.596469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1504/2875] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:14 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:14.633201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:14.633816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:14.640428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:14.640939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:14.641394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:14.642698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:14.643150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:14.643573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:14.770632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:14.779056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-960316d0-a972-4467-b3bd-38253178b9ca tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149776 Aug 24 10:37:14.779608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1372/2876] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:14 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:15.168254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:15.168784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:15.175342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:15.175831 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:15.176301 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:15.176992 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:15.177450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:15.177878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:15.531399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:15.540064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-12c02bfc-3f58-4166-b713-3fa6b2cd92ac tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.375822 Aug 24 10:37:15.540624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1505/2877] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:15 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:15.617019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:15.617633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:15.623600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:15.624110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:15.624550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:15.625230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:15.625686 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:15.626111 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:15.687804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:15.695989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-13c03c61-9818-44d6-bf0c-06526183943b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082907 Aug 24 10:37:15.696575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1373/2878] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:15 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:15.801736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:15.802352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:15.808348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:15.808827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:15.809262 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:15.809939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:15.810390 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:15.810812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:15.940714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:15.948817 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5427a79b-f80e-4ead-a518-51df49b1d283 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150981 Aug 24 10:37:15.949383 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1506/2879] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:15 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:16.565506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:16.566780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:16.572911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:16.573386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:16.573815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:16.574486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:16.574935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:16.575350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:16.703004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:16.711095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-838b690c-a3a1-49a3-b713-c72af50f450c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149474 Aug 24 10:37:16.711651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1374/2880] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:16 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:16.717173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:16.717807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:16.724321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:16.724842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:16.725284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:16.725981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:16.726443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:16.726871 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:16.792788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:16.796203 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:16.802141 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c8b9a97a-93f7-4b24-8cf5-bd9f46e941ee tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.088798 Aug 24 10:37:16.802701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1507/2881] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:16 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:16.970890 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:16.971538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:16.977487 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:16.977966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:16.978399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:16.979075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:16.979527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:16.979949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:17.107118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:17.115978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0843f21e-fc70-4ebb-9029-2adc241a4348 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149318 Aug 24 10:37:17.116938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1375/2882] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:16 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:17.733033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:17.733651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:17.741162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:17.741651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:17.742091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:17.742781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:17.743240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:17.743669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:17.822874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:17.823492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:17.830449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:17.830939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:17.831379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:17.832092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:17.832553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:17.832984 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:17.871049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:17.879594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-64cf5e14-1a9a-4fe2-a8d5-eb519f0055d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150306 Aug 24 10:37:17.880327 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1508/2883] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:17 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:17.897229 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:17.905407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f86dc5f6-8a27-4c7b-adc6-77d928dd9d37 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086421 Aug 24 10:37:17.905968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1376/2884] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:17 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:18.137487 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:18.138099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:18.144547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:18.145034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:18.145472 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:18.146156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:18.146616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:18.147293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:18.276424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:18.285286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9090fc3f-86a5-4c8c-a024-c45e9c105922 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151666 Aug 24 10:37:18.285851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1509/2885] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:18 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:18.901543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:18.902161 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:18.909302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:18.909787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:18.910223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:18.910899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:18.911356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:18.911778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:18.926829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:18.927499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:18.933845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:18.934329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:18.940915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:18.941695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:18.942173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:18.942586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:19.006389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:19.014570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-309c4a04-dacb-487f-b289-c24fcdaa8277 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.091592 Aug 24 10:37:19.015137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1510/2886] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:18 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:19.045053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:19.054075 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-46460871-8da0-4461-8742-cdfa47bebeb6 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.156869 Aug 24 10:37:19.054633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1377/2887] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:18 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:19.306784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:19.307314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:19.313413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:19.313895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:19.314503 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:19.315189 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:19.315636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:19.316084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:19.444455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:19.452987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0e6b68f0-520d-42a1-88ef-246cb0ceefc5 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150137 Aug 24 10:37:19.453563 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1511/2888] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:19 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:20.036330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:20.036889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:20.042980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:20.043463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:20.043920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:20.044672 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:20.045126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:20.045549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:20.077104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:20.077884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:20.084491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:20.084982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:20.085460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:20.086154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:20.086616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:20.087046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:20.106218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:20.115200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ebfa13bd-adef-41ed-b76b-de4315552565 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083516 Aug 24 10:37:20.115774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1378/2889] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:20 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:20.218309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:20.227002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4569ed07-4a63-45d5-9fa9-f76e8e06aad7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153782 Aug 24 10:37:20.227558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1512/2890] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:20 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:20.475315 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:20.475958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:20.482633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:20.483126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:20.483581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:20.484302 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:20.484758 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:20.485184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:20.610615 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:20.618960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-316b6a8e-9c29-4661-85a3-2fa42a724537 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148203 Aug 24 10:37:20.619515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1379/2891] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:20 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:21.135517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:21.136153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:21.143084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:21.143560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:21.143994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:21.144697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:21.145146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:21.145568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:21.206143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:21.214194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c6b5efd9-668d-4a54-9586-72eccb3a3870 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082519 Aug 24 10:37:21.214746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1513/2892] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:21 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:21.247791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:21.248439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:21.254874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:21.255359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:21.255802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:21.256531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:21.256988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:21.257413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:21.388108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:21.397006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1f230c59-909b-423a-95c4-d4fbebcf2535 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153045 Aug 24 10:37:21.397831 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1380/2893] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:21 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:21.640480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:21.641166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:21.647797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:21.648318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:21.648759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:21.649441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:21.649896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:21.650319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:21.774307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:21.782523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ac373f0e-bd37-4195-bcfc-35c94b7ab57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.145922 Aug 24 10:37:21.783087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1514/2894] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:21 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:22.235805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:22.237142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:22.238981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=78 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:22.244620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:22.245103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:22.245539 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:22.246227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:22.246680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:22.247105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:22.307946 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:22.311344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:22.317631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b4e71fa7-b4c1-4f8e-89cb-08d22fe9dad6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086250 Aug 24 10:37:22.318205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1381/2895] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:22 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:22.418155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:22.418768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:22.427555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:22.428039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:22.428498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:22.429179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:22.429631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:22.430055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:22.567083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:22.575227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f97fd7a4-163d-43cf-86d6-6668ef4a5e49 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.160980 Aug 24 10:37:22.575774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1515/2896] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:22 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:22.803847 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:22.804481 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:22.810383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:22.810864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:22.811295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:22.811968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:22.812448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:22.812871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:22.953902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:22.961945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-945565fa-e9a7-43c8-ab29-7034a50fdbf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.162929 Aug 24 10:37:22.962502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1382/2897] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:22 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:23.339271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:23.339882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:23.346637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:23.347121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:23.347557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:23.348286 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:23.348738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:23.349163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:23.409738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:23.417884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-295db5aa-844c-4495-b597-29fa67d7f199 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082476 Aug 24 10:37:23.418378 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1516/2898] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:23 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:23.596200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:23.596817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:23.603276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:23.603761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:23.604228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:23.604912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:23.605386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:23.605798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:23.732401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:23.740630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8f5821e1-ea5c-4bc0-bcd6-3ef78a3f412a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148419 Aug 24 10:37:23.741185 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1383/2899] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:23 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:23.983524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:23.984072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:23.990964 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:23.991443 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:23.991879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:23.992590 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:23.993043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:23.993462 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:24.119937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:24.128673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0fd35985-a3a8-4d7d-bc19-0038762315ce tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149180 Aug 24 10:37:24.129245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1517/2900] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:23 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:24.439122 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:24.439782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:24.445845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:24.446325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:24.446760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:24.447437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:24.447888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:24.448340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:24.509430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:24.517432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f13d4e73-b5cb-461b-a1f6-c4798dc7d000 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082198 Aug 24 10:37:24.518142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1384/2901] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:24 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:24.762038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:24.762594 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:24.769060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:24.769549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:24.769988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:24.770680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:24.771140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:24.771562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:24.906648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:24.915305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2da2b17c-32ec-41f4-94a6-4da302b27526 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.157298 Aug 24 10:37:24.915860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1518/2902] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:24 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:25.150824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:25.151362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:25.157898 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:25.158377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:25.158815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:25.159503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:25.159952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:25.160406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:25.291963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:25.300064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5b752005-52ec-4cad-9aae-da0f4df72f01 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153687 Aug 24 10:37:25.300628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1385/2903] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:25 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:25.539107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:25.539721 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:25.546159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:25.546642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:25.547079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:25.547761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:25.548247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:25.548683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:25.624980 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:25.633793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a480fab9-321e-45f5-b642-599e66e8b351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.098802 Aug 24 10:37:25.634358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1519/2904] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:25 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:25.935936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:25.936583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:25.942581 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:25.943061 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:25.943498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:25.944202 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:25.944657 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:25.945077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:26.072784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:26.080875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dce08427-1acf-466f-ab23-514e4a477a36 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148831 Aug 24 10:37:26.081426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1386/2905] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:25 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:26.320121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:26.320658 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:26.322456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=82 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:26.327675 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:26.328181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:26.328614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:26.329293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:26.329738 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:26.330160 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:26.461727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:26.470960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2410d501-78f9-406e-b0cf-ca095de9ead0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154788 Aug 24 10:37:26.472512 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1520/2906] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:26 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:26.655629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:26.656184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:26.662623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:26.663097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:26.663526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:26.664235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:26.664680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:26.665097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:26.725329 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:26.733549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f22cec6c-5ac4-4b4a-bfb9-c36ea16aad78 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.081865 Aug 24 10:37:26.734115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1387/2907] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:26 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:27.102177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:27.102791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:27.109196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:27.109685 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:27.110215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:27.110906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:27.111360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:27.111786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:27.242735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:27.250890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-953df860-82cb-4f25-b3d5-44ea6f1c7c88 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152603 Aug 24 10:37:27.251456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1521/2908] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:27 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:27.494713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:27.495320 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:27.501193 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:27.501666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:27.502092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:27.502809 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:27.503255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:27.503670 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:27.634455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:27.642733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-aa706a36-77e9-4ffc-8bad-6050a47aea9f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151932 Aug 24 10:37:27.643294 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1388/2909] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:27 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:27.754418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:27.754955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:27.760850 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:27.761331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:27.761762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:27.762438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:27.762885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:27.763305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:27.823785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:27.832422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-941bad46-b06d-431d-9f6e-e12bd2b2fce7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.081923 Aug 24 10:37:27.832972 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1522/2910] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:27 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:28.272957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:28.273834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:28.280238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:28.280720 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:28.281157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:28.281835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:28.282287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:28.282708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:28.407886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:28.416138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fb25541e-895c-4ed3-aa52-700d5855690b tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.147715 Aug 24 10:37:28.416702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1389/2911] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:28 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:28.663727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:28.664289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:28.670699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:28.671213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:28.671639 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:28.672354 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:28.672911 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:28.673436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:28.805492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:28.813945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0d3e4f13-6f24-4bb2-b472-e4d553e46f7f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154178 Aug 24 10:37:28.814502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1523/2912] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:28 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:28.853402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:28.854035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:28.859917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:28.860435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:28.860866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:28.861538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:28.861988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:28.862987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:28.922334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:28.930999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1fb024df-1c5a-4634-84cc-129d146d90e5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.081506 Aug 24 10:37:28.931555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1390/2913] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:28 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:29.437335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:29.437948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:29.445096 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:29.445589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:29.446032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:29.446723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:29.447183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:29.447611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:29.465716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:29.577116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:29.586683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c4e71365-4c73-4368-bbfe-0090b8258d86 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153287 Aug 24 10:37:29.587244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1524/2914] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:29 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:29.834864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:29.835503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:29.842099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:29.842584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:29.843024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:29.843733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:29.844217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:29.844639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:29.951794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:29.952656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:29.960613 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:29.961154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:29.961603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:29.962313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:29.962769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:29.963253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:29.982550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:29.991732 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2b6cd9f9-4e45-49d2-ab72-e620d2f1c8fa tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.160685 Aug 24 10:37:29.992335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1391/2915] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:29 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:30.033938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:30.042966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d3985e9d-3111-4e3d-9e89-3287afbb3f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.094992 Aug 24 10:37:30.043535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1525/2916] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:29 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:30.607675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:30.608233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:30.614374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:30.614862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:30.615296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:30.615976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:30.616459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:30.616880 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:30.745231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:30.754102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fe59a870-e88d-48e6-8446-86a2b6aa1372 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150392 Aug 24 10:37:30.754671 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1392/2917] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:30 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:31.012672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:31.013284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:31.019806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:31.020325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:31.020893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:31.021583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:31.022037 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:31.022466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:31.063773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:31.064395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:31.070550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:31.071033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:31.071465 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:31.072190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:31.072642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:31.073067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:31.137097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:31.145632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-31309d60-4fae-40b0-90e7-c82efea0c3da tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085686 Aug 24 10:37:31.146212 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1393/2918] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:31 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:31.160646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:31.168808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5ac899ab-b358-4a66-a6c8-29e65d5c1d11 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.160045 Aug 24 10:37:31.169363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1526/2919] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:31 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:31.775088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:31.775716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:31.781644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:31.782120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:31.782554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:31.783236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:31.783682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:31.784127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:31.914757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:31.922858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6246eb85-eff6-4f54-bfe5-6e6f9898d246 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151948 Aug 24 10:37:31.923413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1394/2920] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:31 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:32.166644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:32.167207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:32.173552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:32.174032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:32.174467 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:32.175815 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:32.176305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:32.176736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:32.189123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:32.189731 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:32.196156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:32.196643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:32.197081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:32.197772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:32.198225 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:32.198649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:32.238244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:32.246481 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d01836cb-2ed5-4a6e-8d8a-6f84c6692ef6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083780 Aug 24 10:37:32.247042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1527/2921] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:32 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:32.327230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:32.336428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-60abc327-e2d4-4aaa-9af4-e3cbf4bb0191 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151119 Aug 24 10:37:32.336993 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1395/2922] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:32 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:32.944625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:32.945274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:32.951249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:32.951739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:32.952225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:32.952916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:32.953374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:32.953814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:33.079867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:33.088007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bea7e7e2-c6ce-4d75-a9d6-f1c044691127 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.147984 Aug 24 10:37:33.088586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1528/2923] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:32 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:33.266641 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:33.267242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:33.273658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:33.274405 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:33.274834 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:33.275532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:33.275979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:33.276431 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:33.340683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:33.349241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8a3cef31-b276-4123-a143-4fc29341f1c6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086435 Aug 24 10:37:33.349799 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1396/2924] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:33 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:33.358099 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:33.358816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:33.365641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:33.366130 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:33.366567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:33.367253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:33.367990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:33.368552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:33.501499 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:33.510185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2a9eb2b1-8e97-4055-ad0c-bb2b967592e7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.155984 Aug 24 10:37:33.510743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1529/2925] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:33 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:34.108998 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:34.109526 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:34.115738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:34.116258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:34.116699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:34.117380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:34.117837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:34.118261 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:34.244112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:34.252279 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bfac41b9-35e2-4967-b74e-7fb7b6b84381 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.147267 Aug 24 10:37:34.252850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1397/2926] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:34 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:34.370148 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:34.370947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:34.377426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:34.377916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:34.378868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:34.379557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:34.380022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:34.380478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:34.442062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:34.450758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b9c37fe0-135a-41d1-9c22-8474046e0f39 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084597 Aug 24 10:37:34.451311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1530/2927] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:34 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:34.532004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:34.532697 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:34.539318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:34.539780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:34.540254 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:34.540929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:34.541380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:34.541801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:34.559836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:34.688323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:34.697067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9fed3a18-e022-4fc1-8a3b-79fdd7e84023 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.168945 Aug 24 10:37:34.697621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1398/2928] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:34 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:35.274625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:35.275256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:35.281201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:35.281681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:35.282111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:35.282784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:35.283231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:35.283649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:35.411155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:35.419228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-200ebfbe-ac45-45b1-9250-9b5617a6d4c8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.148807 Aug 24 10:37:35.419775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1531/2929] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:35 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:35.470737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:35.471351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:35.478301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:35.478865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:35.479323 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:35.480030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:35.480519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:35.480976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:35.544451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:35.552602 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3784d04f-c470-477b-a056-b86b8dde7ff9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085725 Aug 24 10:37:35.553166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1399/2930] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:35 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:35.718176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:35.718783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:35.724683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:35.725173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:35.725608 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:35.726283 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:35.726731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:35.727152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:35.857411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:35.865502 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9fd97b23-c54c-4f5b-b472-dde1acf7579e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151234 Aug 24 10:37:35.866062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1532/2931] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:35 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:36.440441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:36.441112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:36.447072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:36.447557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:36.447996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:36.448705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:36.449158 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:36.449583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:36.573490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:36.574088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:36.578693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:36.580542 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:36.581031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:36.581475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:36.582176 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:36.582640 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:36.583070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:36.586915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e026a8c9-828b-4d2b-af77-18bdf68e28e4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150381 Aug 24 10:37:36.587475 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1400/2932] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:36 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:36.645718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:36.654107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-096091b0-f3c4-48e0-889a-8ee4037a7be2 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084458 Aug 24 10:37:36.654680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1533/2933] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:36 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:36.887359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:36.887972 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:36.893981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:36.894949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:36.895382 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:36.896073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:36.896554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:36.896963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:37.030417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:37.039466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-65ee0393-90c9-40b3-8950-e0d257c7f760 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.156068 Aug 24 10:37:37.040018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1401/2934] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:36 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:37.609101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:37.609718 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:37.616070 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:37.616577 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:37.617022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:37.617707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:37.618163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:37.618588 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:37.675893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:37.676611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:37.683006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:37.683493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:37.683939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:37.684668 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:37.685123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:37.685550 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:37.750374 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:37.752249 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:37.760131 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b650d434-c767-48a5-af7c-5674f0492679 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154781 Aug 24 10:37:37.760782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-34553fe8-c48b-4542-8856-6a3419def714 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.088755 Aug 24 10:37:37.761004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1534/2935] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:37 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:37.763514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1402/2936] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:37 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:38.059516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:38.060161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:38.066745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:38.067228 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:38.067670 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:38.068392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:38.068848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:38.069272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:38.200466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:38.208587 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dab75fb4-7456-4d80-9fa0-ca56e07e9b38 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153017 Aug 24 10:37:38.209145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1535/2937] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:38 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:38.792627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:38.793190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:38.793603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:38.793720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:38.799907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:38.800181 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:38.800428 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:38.800648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:38.800861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:38.801085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:38.801548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:38.801779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:38.802000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:38.802286 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:38.803067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:38.803242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:38.865424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:38.873923 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7fda4e0e-d663-4e36-8ccc-1e0b5630a143 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084576 Aug 24 10:37:38.874488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1536/2938] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:38 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:38.933134 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:38.941741 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6b5ccd04-b9c7-4cbc-a5f6-980ad166a789 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153307 Aug 24 10:37:38.942301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1403/2939] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:38 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:39.230187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:39.230801 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:39.236743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:39.237230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:39.237673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:39.238387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:39.238842 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:39.239269 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:39.377272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:39.385752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-212b18fc-9e2a-4c31-bd15-9d17a0174c8e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.159470 Aug 24 10:37:39.386320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1537/2940] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:39 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:39.894557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:39.895182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:39.902442 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:39.902917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:39.903349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:39.904021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:39.904497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:39.904939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:39.963826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:39.964299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:39.965782 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:39.971242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:39.971805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:39.972271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:39.972963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:39.973445 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:39.973872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:39.974021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ce328216-c4a3-4b57-b1eb-55be8c94b855 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083195 Aug 24 10:37:39.975146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1404/2941] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:39 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:40.100981 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:40.109209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f2152b88-adbb-48b3-98b7-38d663cd8c15 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149494 Aug 24 10:37:40.109770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1538/2942] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:39 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:40.408568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:40.409100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:40.414989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:40.415477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:40.415940 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:40.416648 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:40.417108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:40.417530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:40.549143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:40.557848 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5ae027f1-17d3-4513-a80e-34eeb1fc7fc7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153253 Aug 24 10:37:40.558456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1405/2943] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:40 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:40.995323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:40.995918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:41.002308 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:41.002786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:41.003221 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:41.003896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:41.004387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:41.004807 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:41.065799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:41.076274 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9f541068-a2bd-497d-8ecd-fb650d8bf474 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084349 Aug 24 10:37:41.077295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1539/2944] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:40 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:41.129850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:41.130455 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:41.137031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:41.137548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:41.137983 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:41.138721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:41.139174 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:41.139598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:41.266769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:41.275899 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b56d5be2-6f93-4306-b025-2cf1ce7146c5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149863 Aug 24 10:37:41.276490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1406/2945] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:41 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:41.579774 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:41.580425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:41.587076 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:41.587559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:41.588002 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:41.588711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:41.589165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:41.589595 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:41.719454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:41.727759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4fc7b62e-8b4f-4c15-ac55-4aff60408268 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151843 Aug 24 10:37:41.728365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1540/2946] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:41 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:42.097296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:42.097904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:42.103937 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:42.104452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:42.104915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:42.105598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:42.106050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:42.106473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:42.166477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:42.174558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-67a39154-f528-4c54-8e4d-1ee3b0722185 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.081160 Aug 24 10:37:42.175118 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1407/2947] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:42 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:42.296442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:42.297089 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:42.304230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:42.304736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:42.305158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:42.305846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:42.306300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:42.306723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:42.310703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:42.434313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:42.442429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3e83fefd-0f40-4ef7-a97d-772e5e2e51c0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149837 Aug 24 10:37:42.442990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1541/2948] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:42 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:42.748872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:42.749478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:42.755597 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:42.756102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:42.756611 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:42.757223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:42.757680 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:42.758085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:42.887699 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:42.895862 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-82c2ad79-b4d8-43a1-bd58-4ff61d5b83e4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150919 Aug 24 10:37:42.896952 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1408/2949] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:42 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:43.196433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:43.197042 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:43.202926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:43.203403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:43.203835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:43.204536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:43.204985 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:43.205409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:43.266458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:43.275122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-22e50a52-7cc9-411c-ab25-d84555bd7b97 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083047 Aug 24 10:37:43.275953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1542/2950] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:43 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:43.462778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:43.463341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:43.469679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:43.470154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:43.470582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:43.471281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:43.471712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:43.472418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:43.603296 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:43.611378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-32206a6e-9753-4e1d-b2d5-a3c4af300700 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152576 Aug 24 10:37:43.611979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1409/2951] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:43 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:43.918201 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:43.918808 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:43.925305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:43.925784 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:43.926219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:43.926901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:43.927349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:43.928456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:44.057759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:44.065916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-02f3a47c-7151-4ea3-8316-470a47d14631 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151601 Aug 24 10:37:44.066471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1543/2952] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:43 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:44.297531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:44.298136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:44.304740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:44.305230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:44.305828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:44.306595 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:44.306957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:44.307375 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:44.370803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:44.379002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-accedfdd-2779-4989-a5a6-ad35e25d5f9c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085773 Aug 24 10:37:44.379562 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1410/2953] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:44 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:44.633617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:44.634229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:44.640177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:44.640666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:44.641105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:44.641791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:44.642246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:44.642672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:44.771011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:44.779554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-76808596-979a-4be0-a60e-f9fe331d52d4 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150376 Aug 24 10:37:44.780125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1544/2954] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:44 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:45.086682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:45.087282 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:45.093812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:45.094298 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:45.094733 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:45.095712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:45.096439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:45.096634 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:45.226476 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:45.234964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a9130944-6b71-424c-9424-295c2da562cb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152167 Aug 24 10:37:45.235533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1411/2955] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:45 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:45.399295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:45.399907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:45.406611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:45.407092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:45.407529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:45.408239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:45.408695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:45.409122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:45.472282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:45.481531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1b2fda9e-aa22-4af1-971f-770fa59bf7e0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086071 Aug 24 10:37:45.482101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1545/2956] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:45 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:45.803449 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:45.804534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:45.812849 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:45.813335 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:45.813770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:45.814457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:45.814904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:45.815353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:45.942815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:45.950949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4add9772-6186-459a-9945-287a69c7c847 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153209 Aug 24 10:37:45.951511 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1412/2957] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:45 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:46.258461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:46.259082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:46.265017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:46.265505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:46.265947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:46.266643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:46.267100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:46.267529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:46.397469 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:46.406319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-03b8d522-431e-44ac-b5da-102df8ae3a5f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152295 Aug 24 10:37:46.406921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1546/2958] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:46 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:46.503909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:46.504690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:46.512609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:46.513066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:46.513505 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:46.514200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:46.514681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:46.515154 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:46.576573 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:46.584679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-67736f4f-b6f2-49c9-81bd-a8b5c6ba26bf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085877 Aug 24 10:37:46.585231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1413/2959] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:46 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:46.972466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:46.973082 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:46.979034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:46.979527 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:46.979973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:46.980704 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:46.981161 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:46.981592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:47.111006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:47.120706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d994fb2a-eaca-42e4-9c8f-ccab4b9dc60a tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152225 Aug 24 10:37:47.121270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1547/2960] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:46 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:47.427792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:47.428832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:47.434780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:47.435262 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:47.435695 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:47.436403 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:47.436859 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:47.437283 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:47.568628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:47.576883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e9ff827f-fc53-446d-a76a-494280e12728 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153095 Aug 24 10:37:47.577448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1414/2961] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:47 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:47.605614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:47.606225 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:47.612672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:47.613154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:47.613596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:47.614276 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:47.614731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:47.615159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:47.676239 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:47.684859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9fc3f690-98a1-4708-96f0-418af229172c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083022 Aug 24 10:37:47.685425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1548/2962] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:47 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:48.142238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:48.142756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:48.149198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:48.149676 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:48.150105 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:48.150776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:48.151486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:48.151902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:48.155801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:48.280080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:48.288308 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ec74cead-90ab-41b9-83db-dc3c84d52459 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.149970 Aug 24 10:37:48.288871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1415/2963] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:48 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:48.598206 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:48.598812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:48.604699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:48.605179 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:48.605616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:48.606294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:48.606845 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:48.607149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:48.705726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:48.706340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:48.712703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:48.713189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:48.713626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:48.714316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:48.714766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:48.715195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:48.739657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:48.748406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-90d2472c-8bae-429b-8bf4-72d30c909b7c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154110 Aug 24 10:37:48.748978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1549/2964] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:48 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:48.780613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:48.790053 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2031d37c-92be-423f-8160-ec23fee4e838 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.088052 Aug 24 10:37:48.790621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1416/2965] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:48 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:49.309942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:49.310553 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:49.316734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:49.317209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:49.317651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:49.318368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:49.318876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:49.319285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:49.444783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:49.452897 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-922953d4-b4a0-4a57-b592-75bf60896697 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.146906 Aug 24 10:37:49.453460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1550/2966] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:49 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:49.771316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:49.771939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:49.779160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:49.779645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:49.780106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:49.780804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:49.781263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:49.781693 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:49.810583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:49.811207 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:49.817927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:49.818405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:49.819372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:49.820022 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:49.820495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:49.820912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:49.886707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:49.895023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-95b32e83-20d2-4f96-b8aa-1937c22f2980 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.088181 Aug 24 10:37:49.907551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1551/2967] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:49 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:49.910034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:49.918684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8289d26b-3b51-4d78-adb2-ce9bb1e74a92 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.152152 Aug 24 10:37:49.919256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1417/2968] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:49 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:50.474559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:50.475173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:50.481865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:50.482363 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:50.482805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:50.483506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:50.483965 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:50.484491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:50.613995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:50.622567 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bb10f90e-23ab-4c0d-a94c-8481ec67b5c7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151958 Aug 24 10:37:50.623139 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1552/2969] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:50 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:50.916502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:50.917113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:50.923484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:50.923962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:50.924426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:50.925204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:50.925656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:50.926082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:50.938836 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:50.939440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:50.945765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:50.946243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:50.946681 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:50.947362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:50.947814 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:50.948265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:50.990694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:50.998850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6d6621a2-e4ff-4daa-9b1d-c3f0214bf072 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086908 Aug 24 10:37:50.999415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1418/2970] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:50 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:51.075995 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:51.084137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-38ffbd43-7ea6-43e0-bb36-1e20bd818f4e tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.149233 Aug 24 10:37:51.085187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1553/2971] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:50 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:51.643394 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:51.644230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:51.651243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:51.651997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:51.652462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:51.653148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:51.653609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:51.654036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:51.783909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:51.792762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c0c28430-322e-4134-bca7-af985b2dfea7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153183 Aug 24 10:37:51.793332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1419/2972] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:51 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:52.019865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:52.020500 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:52.027124 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:52.027609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:52.028061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:52.028755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:52.029213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:52.029638 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:52.093517 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:52.102353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5c5d68a1-bb21-40bc-bd7b-84de5d770108 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086365 Aug 24 10:37:52.102914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1554/2973] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:52 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:52.107248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:52.107854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:52.114304 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:52.114779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:52.115209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:52.115884 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:52.116927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:52.117346 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:52.243509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:52.251616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-55fa0e9a-f008-46a2-a525-9e0cd448e57b tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148334 Aug 24 10:37:52.252208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1420/2974] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:52 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:52.814461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:52.815104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:52.822079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:52.822568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:52.823013 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:52.823701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:52.824192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:52.824626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:52.952578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:52.961470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d385d3ba-600f-4d22-89db-779ab1e395f8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150884 Aug 24 10:37:52.962057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1555/2975] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:52 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:53.122751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:53.123316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:53.125580 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=80 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:53.130742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:53.131215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:53.132152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:53.132829 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:53.133311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:53.133730 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:53.195443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:53.203547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-221a5ec0-462b-4d0a-a008-18224e28650d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084751 Aug 24 10:37:53.204113 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1421/2976] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:53 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:53.272348 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:53.272961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:53.279584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:53.280086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:53.280525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:53.281197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:53.281644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:53.282064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:53.410614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:53.418942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-af98e066-c6a3-48ed-a5fa-f3dce4c9dd95 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150553 Aug 24 10:37:53.419516 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1556/2977] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:53 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:53.983636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:53.984309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:53.990268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:53.990756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:53.991222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:53.991931 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:53.992415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:53.992843 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:54.122705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:54.131146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-558738b7-f4e7-43ae-aa6c-8dc94e79bab8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151808 Aug 24 10:37:54.131787 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1422/2978] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:53 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:54.224311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:54.224951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:54.231593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:54.232101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:54.232541 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:54.233222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:54.233677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:54.234103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:54.297066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:54.300666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:54.306320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ddc9f92b-1540-4a39-841f-3c19cad04c65 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085862 Aug 24 10:37:54.306878 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1557/2979] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:54 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:54.440644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:54.441254 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:54.447616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:54.448115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:54.448548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:54.449227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:54.449679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:54.450099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:54.577198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:54.585930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f1a18fab-9796-4247-b4bb-a7cee05ede32 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148521 Aug 24 10:37:54.586480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1423/2980] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:54 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:55.153163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:55.153693 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:55.159810 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:55.160323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:55.160768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:55.161459 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:55.161919 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:55.162352 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:55.292533 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:55.300959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d3379aef-4805-4868-95c9-5a89d7dd2071 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152076 Aug 24 10:37:55.301556 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1558/2981] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:55 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:55.327678 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:55.328327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:55.334751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:55.335231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:55.335665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:55.336380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:55.336833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:55.337255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:55.397223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:55.406101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2e6780af-b849-44fe-94e4-00ad74898f0a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082264 Aug 24 10:37:55.406607 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1424/2982] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:55 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:55.606791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:55.607407 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:55.613324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:55.613794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:55.614224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:55.614898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:55.615343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:55.615760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:55.743194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:55.751367 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-53398be5-da1c-40d4-bca2-51bb7d57dd00 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148510 Aug 24 10:37:55.751934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1559/2983] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:55 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:56.322675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:56.323284 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:56.330051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:56.330636 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:56.330951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:56.331635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:56.332102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:56.332524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:56.426637 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:56.428825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:56.435191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=80 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:37:56.437025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:56.437583 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:56.438100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:56.439133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:56.439663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:56.440191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:56.466583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:56.475378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2457561b-a9a9-490c-a419-2ab4af3e46b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.156500 Aug 24 10:37:56.476557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1425/2984] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:56 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:56.507997 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:56.516628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-113c196b-4a3a-41aa-8b7f-25529f43222b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.093891 Aug 24 10:37:56.517186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1560/2985] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:56 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:56.773957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:56.774600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:56.780824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:56.781311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:56.781749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:56.782464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:56.782925 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:56.783349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:56.912184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:56.920533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-811c9993-177b-4b4f-b487-e966c8b5bcf2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151197 Aug 24 10:37:56.921110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1426/2986] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:56 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:57.497838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:57.498447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:57.504959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:57.505446 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:57.505885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:57.506571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:57.507027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:57.507456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:57.537879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:57.539018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:57.546138 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:57.546629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:57.547066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:57.547759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:57.548237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:57.548660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:57.610276 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:57.619610 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7b24b158-ba27-49e4-92d3-7c0d636abd8c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.085231 Aug 24 10:37:57.620379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1427/2987] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:57 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:57.637411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:57.645491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3a924da3-97e0-41bd-90cc-9aa975581613 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151540 Aug 24 10:37:57.646043 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1561/2988] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:57 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:57.942034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:57.942649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:57.948603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:57.949087 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:57.949525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:57.950205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:57.950667 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:57.951092 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:58.079761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:58.088535 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0bcbd690-c694-46f6-bb65-6791b93f7d41 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150317 Aug 24 10:37:58.089097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1428/2989] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:57 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:58.640392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:58.641008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:58.647433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:58.647916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:58.648387 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:58.649072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:58.649531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:58.649960 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:58.666964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:58.667584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:58.674143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:58.674643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:58.675063 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:58.676763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:58.676763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:58.676763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:58.711758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:58.720009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c14d2be2-1826-4c20-8d57-d8a98e2c50dc tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.083527 Aug 24 10:37:58.720618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1562/2990] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:58 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:58.806590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:58.814897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-51a9864b-fcaa-4870-b29b-deddb2198588 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.151940 Aug 24 10:37:58.815469 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1429/2991] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:58 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:37:59.109733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:59.110342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:59.117369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:59.117849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:59.118290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:59.118975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:59.119426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:59.119851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:59.248913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:59.257195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-21bfcbd4-6e8e-4471-bbb4-709b23d268ae tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.151441 Aug 24 10:37:59.257756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1563/2992] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:59 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:59.740915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:59.741564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:59.747964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:59.748477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:59.748917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:59.749604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:59.750058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:59.750480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:59.815009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:59.823866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b5fb536a-e0b2-4c98-94df-bc19ee37400b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086818 Aug 24 10:37:59.824399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1430/2993] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:59 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:37:59.835332 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:37:59.836560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:37:59.842986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:59.843478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:59.843921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:59.844657 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:37:59.845128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:37:59.845581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:37:59.976294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:37:59.984484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d080de0c-191f-42ad-95d2-5128b626bc1c tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.152966 Aug 24 10:37:59.985046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1564/2994] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:37:59 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:00.278850 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:00.279503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:00.285507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:00.285990 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:00.286429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:00.287115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:00.287571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:00.288000 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:00.413452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:00.417423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:00.422924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5f6bf832-77ea-4995-bd08-41b92a652529 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.148617 Aug 24 10:38:00.423498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1431/2995] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:00 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:00.845144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:00.845762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:00.853856 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:00.854356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:00.854819 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:00.855518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:00.855986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:00.856448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:00.917372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:00.925597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3415a6ea-dcda-47c4-b9ae-0e53e51d0bd1 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.084324 Aug 24 10:38:00.926155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1565/2996] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:00 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:01.005944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:01.006583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:01.013960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:01.014444 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:01.014883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:01.015570 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:01.016028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:01.016480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:01.143451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:01.152348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-27432f19-deb4-483b-988f-85f2ebdcba60 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150216 Aug 24 10:38:01.153191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1432/2997] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:01 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:01.444707 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:01.445978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:01.451768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:01.452280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:01.452714 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:01.453392 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:01.453843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:01.454266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:01.585677 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:01.593987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6c4b35b0-084f-41bb-afb1-f1a06516e339 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.153227 Aug 24 10:38:01.594550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1566/2998] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:01 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:01.946389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:01.946985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:01.952891 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:01.953368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:01.953794 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:01.954463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:01.954906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:01.955321 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:02.025439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:02.034050 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-74a31401-cd23-4739-acc2-795149a89c2b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.091535 Aug 24 10:38:02.034617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1433/2999] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:01 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:02.173865 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:02.174498 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:02.181309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:02.181802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:02.182244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:02.182948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:02.183406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:02.183838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:02.314781 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:02.323005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-27ba6617-ca1e-4593-b380-036981049902 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.153149 Aug 24 10:38:02.323603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1567/3000] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:02 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:02.615127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:02.615656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:02.621687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:02.622163 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:02.622593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:02.623271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:02.623719 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:02.624170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:02.752768 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:02.761362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-55356fa0-9d61-49f8-9a73-e5ac7c11f4c8 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.150264 Aug 24 10:38:02.761936 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1434/3001] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:02 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:03.055232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:03.055840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:03.061791 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:03.062267 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:03.062699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:03.063375 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:03.063826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:03.064277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:03.125011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:03.133629 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-24968987-8485-4569-bf47-3f77a669e02a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.082284 Aug 24 10:38:03.134181 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1568/3002] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:03 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:03.344930 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:03.345514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:03.353328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:03.353824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:03.354265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:03.354981 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:03.355440 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:03.355871 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:03.496309 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:03.504619 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1f9e97ad-6273-4f60-802a-aafee41faae7 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.163761 Aug 24 10:38:03.505187 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1435/3003] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:03 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:03.782904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:03.783529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:03.790715 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:03.791205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:03.791646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:03.792373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:03.792834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:03.793271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:03.924649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:03.933066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bf7e78a2-a7c2-40d4-9135-5683b67bf266 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.154003 Aug 24 10:38:03.933636 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1569/3004] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:03 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:04.154497 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:04.155103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:04.161152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:04.161635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:04.162056 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:04.162729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:04.163176 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:04.163594 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:04.228622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:04.237112 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-289b1adc-8862-452c-b7b0-b9249ebee05b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1368 microversion: 2.1 time: 0.086792 Aug 24 10:38:04.237682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1436/3005] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:04 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1368 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:04.526003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:04.526611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:04.532901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:04.533386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:04.533823 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:04.534501 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:04.534952 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:04.535373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:04.668521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:04.676760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ca26515d-375d-4226-92f2-74f9ce7c76e1 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.154672 Aug 24 10:38:04.677321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1570/3006] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:04 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:04.953805 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:04.954446 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:04.960941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:04.961427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:04.961865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:04.962553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:04.963007 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:04.963436 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:05.104006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:05.116005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4bdb797c-772e-49da-a89e-ce8b4bd5ec78 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.165952 Aug 24 10:38:05.121201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1437/3007] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:04 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:05.258318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:05.258927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:05.265279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:05.265758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:05.266194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:05.266873 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:05.267326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:05.267749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:05.300460 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:05.300954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:05.301401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:05.301922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lazy-loading 'fault' on Instance uuid 5692cc07-3f16-47a4-9993-503b451d9951 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:38:05.348265 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:05.357764 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-83b14fea-534c-4136-9ce7-29012ce824af tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 200 len: 1462 microversion: 2.1 time: 0.103480 Aug 24 10:38:05.358338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1571/3008] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:05 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 1462 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:05.698970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:05.699572 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:05.705592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:05.706068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:05.706493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:05.707167 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:05.707622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:05.708036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:05.837324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:05.845422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2504f311-65bf-410d-b0d4-294e814f5282 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1736 microversion: 2.1 time: 0.150925 Aug 24 10:38:05.845976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1438/3009] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:05 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1736 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:06.138687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:06.142026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:06.155891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:06.156420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:06.156867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:06.157578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:06.158038 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:06.158468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:06.310488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:06.318663 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bd86afed-2e56-411c-bcf0-5320d9879514 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1561 microversion: 2.1 time: 0.184066 Aug 24 10:38:06.319231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1572/3010] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:06 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1561 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:06.378690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:06.379231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:06.385711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:06.386197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:06.386638 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:06.387324 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:06.387780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:06.388241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:06.401554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] HTTP exception thrown: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Request method failure captured: Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 HTTP/1.1 Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfei3ERPSR8lry9h62i-mTjfLEiY_G0pf4TOSE4I4STV_SQYl2_UIncspaIl58uBJJzOdoQ7pLY4XfcLnM6PIi-fCQvX7S1I2xM3rJol0zQr6Uy5AXr20ftWYLN4Ls8-en59yYA6dcGI9oXswbuIZaITfY674PQC2m8u_HkGCNmRiYI Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:38:06.403447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 1de97cce55e944dbb62a920ea11734f5 Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:38:06.405389 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '5692cc07-3f16-47a4-9993-503b451d9951'} Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:38:06.406624 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:38:06.408519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:06.410069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Returning 404 to user: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:38:06.410069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dd2e5cc1-e476-46a5-9109-3e29938c636a tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 404 len: 111 microversion: 2.1 time: 0.030080 Aug 24 10:38:06.410069 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1439/3011] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:06 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:38:06.413568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2098b4e3-6250-4f9d-ae70-a848e617b6fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '.version_select at 0x7fa33d9f2040>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:06.435118 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2098b4e3-6250-4f9d-ae70-a848e617b6fb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1630910318" status: 202 len: 0 microversion: 2.1 time: 0.025296 Aug 24 10:38:06.435694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1573/3012] 10.222.0.86 () {58 vars in 1221 bytes} [Wed Aug 24 10:38:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1630910318 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:38:06.865813 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:06.866660 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:06.872956 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:06.873712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:06.874358 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:06.875231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:06.876585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:06.877192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:07.073830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:07.085038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-73814c44-f7a1-4d95-b78b-decc1f32f5ea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1690 microversion: 2.1 time: 0.222528 Aug 24 10:38:07.086168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1440/3013] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:06 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1690 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:07.106597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-b29587e4-82b4-4120-ba08-3569420f7047 req-44e333a1-6aa8-4c01-b983-4099a307d927 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "987654b5-b3b6-434c-a284-d630ab8ab4ad", "name": "network-vif-deleted", "tag": "98183aae-275c-45c1-b9fc-a7d038275f2d"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:07.130254 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b29587e4-82b4-4120-ba08-3569420f7047 req-44e333a1-6aa8-4c01-b983-4099a307d927 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:07.130758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b29587e4-82b4-4120-ba08-3569420f7047 req-44e333a1-6aa8-4c01-b983-4099a307d927 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:07.131196 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b29587e4-82b4-4120-ba08-3569420f7047 req-44e333a1-6aa8-4c01-b983-4099a307d927 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:07.143422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-b29587e4-82b4-4120-ba08-3569420f7047 req-44e333a1-6aa8-4c01-b983-4099a307d927 service nova] Creating event network-vif-deleted:98183aae-275c-45c1-b9fc-a7d038275f2d for instance 987654b5-b3b6-434c-a284-d630ab8ab4ad on hv-cinder-85427 Aug 24 10:38:07.149523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-b29587e4-82b4-4120-ba08-3569420f7047 req-44e333a1-6aa8-4c01-b983-4099a307d927 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.104933 Aug 24 10:38:07.150204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1574/3014] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:07.341804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:07.343418 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:07.350632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:07.351390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:07.352560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:07.353457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:07.354082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:07.354673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:07.439772 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-157aab30-c83a-4dda-b09c-f20700dfc5e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'create', calling method: .version_select at 0x7fa33d82e5e0>, body: {"keypair": {"name": "tempest-keypair-1355527141", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD9nIzEtW9i+HY59Z4ln5/scRH8WCc4F+y0XSz1LZVgWRrN4G+VeSDlMJHhlcfPNeOsQMJG4HrgB2nefM+N8rk3w0iMa9msaCKMMzYXjpaHZzWzWfKFJpL5hSgVpSjmEZQ=="}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:07.456331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-157aab30-c83a-4dda-b09c-f20700dfc5e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Getting quotas for project e055ba4967db42108c069ebc3843dcf6. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:38:07.460627 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-157aab30-c83a-4dda-b09c-f20700dfc5e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Getting quotas for user 1de97cce55e944dbb62a920ea11734f5 and project e055ba4967db42108c069ebc3843dcf6. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:38:07.488423 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-157aab30-c83a-4dda-b09c-f20700dfc5e7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.053113 Aug 24 10:38:07.489386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1575/3015] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:38:07 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:38:07.521104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:07.530196 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-049ed4c3-592c-49e7-b3ce-1d43ab7a1a9d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1515 microversion: 2.1 time: 0.192780 Aug 24 10:38:07.531030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1441/3016] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:07 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1515 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:08.107092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:08.107830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:08.114696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:08.115192 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:08.115641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:08.116633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:08.117097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:08.117566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:08.533285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:08.537021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=100 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:08.543409 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9ba09579-4b98-4397-b739-bcd346701a40 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1690 microversion: 2.1 time: 0.439907 Aug 24 10:38:08.543996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1576/3017] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:08 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1690 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:08.559674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:08.560328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:08.566864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:08.567353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:08.567790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:08.568512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:08.568966 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:08.569390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:08.709295 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:08.717538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-24214b15-546b-47e5-bf0b-02e31f95b88d tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02" status: 200 len: 1515 microversion: 2.1 time: 0.162697 Aug 24 10:38:08.718219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1442/3018] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:08 2022] GET /compute/v2.1/servers/0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 => generated 1515 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:38:09.566744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:09.568706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:09.575102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:09.575593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:09.576033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:09.576766 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:09.577256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:09.577820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:09.697779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:09.705835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-00ca060e-e8a4-40ac-81c4-3a0e0bdfd37e tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1622 microversion: 2.1 time: 0.142661 Aug 24 10:38:09.706631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1577/3019] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:09 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1622 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:38:09.720429 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-12a3ac2c-5965-41d8-bf21-bb53722677ce req-d62bf48f-f309-43b8-a8e4-618f3565f61c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0c3cf0b5-40f3-40f9-83ec-6de8d63aca02", "name": "network-vif-deleted", "tag": "cac7341d-d892-4b93-a35e-a6dc241272a3"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:09.732084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-12a3ac2c-5965-41d8-bf21-bb53722677ce req-d62bf48f-f309-43b8-a8e4-618f3565f61c service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:09.732631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-12a3ac2c-5965-41d8-bf21-bb53722677ce req-d62bf48f-f309-43b8-a8e4-618f3565f61c service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:09.733178 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-12a3ac2c-5965-41d8-bf21-bb53722677ce req-d62bf48f-f309-43b8-a8e4-618f3565f61c service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:09.747474 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-12a3ac2c-5965-41d8-bf21-bb53722677ce req-d62bf48f-f309-43b8-a8e4-618f3565f61c service nova] Creating event network-vif-deleted:cac7341d-d892-4b93-a35e-a6dc241272a3 for instance 0c3cf0b5-40f3-40f9-83ec-6de8d63aca02 on hv-cinder-85427 Aug 24 10:38:09.752651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-12a3ac2c-5965-41d8-bf21-bb53722677ce req-d62bf48f-f309-43b8-a8e4-618f3565f61c service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036158 Aug 24 10:38:09.753426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1443/3020] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:10.230010 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-433302411", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "65642c38-aa5e-4b7b-9606-eb1b26338156"}], "security_groups": [{"name": "tempest-securitygroup--229999465"}], "key_name": "tempest-keypair-1355527141", "user_data": "}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:10.351558 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] validate_networks() for [('65642c38-aa5e-4b7b-9606-eb1b26338156', None, None, None, None, None)] {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:38:10.687079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:38:10.687775 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:38:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:38:10.730735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:10.731479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:10.734660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:38:10.735111 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:38:10.735536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:38:10.735940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:38:10.736372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:38:10.746462 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:10.746949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:10.747121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:10.747385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:10.747694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:10.748086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:10.748307 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:10.748536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:10.749077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:10.749242 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:10.749795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:10.750336 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:10.768779 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Getting quotas for project e055ba4967db42108c069ebc3843dcf6. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:38:10.773435 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Getting quotas for user 1de97cce55e944dbb62a920ea11734f5 and project e055ba4967db42108c069ebc3843dcf6. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:38:10.791200 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:38:10.793695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:38:10.825340 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:10.833846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6ff424ba-5483-4ce5-9253-e18a86e3b8a5 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1622 microversion: 2.1 time: 0.109018 Aug 24 10:38:10.834518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1444/3021] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:10 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1622 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:10.852406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f72ab81d-0499-477c-bec1-b11ecfc17daf tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.626595 Aug 24 10:38:10.853031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1578/3022] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:38:10 2022] POST /compute/v2.1/servers => generated 401 bytes in 628 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:38:10.865863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8fac8e1a-0c77-4bd9-9887-cc1c13234ba4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:10.866757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8fac8e1a-0c77-4bd9-9887-cc1c13234ba4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:10.924441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8fac8e1a-0c77-4bd9-9887-cc1c13234ba4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:10.932571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8fac8e1a-0c77-4bd9-9887-cc1c13234ba4 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1237 microversion: 2.1 time: 0.071120 Aug 24 10:38:10.933146 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1445/3023] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:10 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1237 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:11.775619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-101b07cc-4203-4691-86bb-7015c520b51e req-83e5b7c2-9f8b-429f-980b-15fb7cbf3248 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bdb467be-865b-489c-ab6c-f714a02672e1", "name": "network-vif-deleted", "tag": "7b852e03-cec0-4436-b50c-9c652193755e"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:11.786025 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-101b07cc-4203-4691-86bb-7015c520b51e req-83e5b7c2-9f8b-429f-980b-15fb7cbf3248 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:11.786508 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-101b07cc-4203-4691-86bb-7015c520b51e req-83e5b7c2-9f8b-429f-980b-15fb7cbf3248 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:11.786945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-101b07cc-4203-4691-86bb-7015c520b51e req-83e5b7c2-9f8b-429f-980b-15fb7cbf3248 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:11.797326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-101b07cc-4203-4691-86bb-7015c520b51e req-83e5b7c2-9f8b-429f-980b-15fb7cbf3248 service nova] Creating event network-vif-deleted:7b852e03-cec0-4436-b50c-9c652193755e for instance bdb467be-865b-489c-ab6c-f714a02672e1 on hv-cinder-85427 Aug 24 10:38:11.802330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-101b07cc-4203-4691-86bb-7015c520b51e req-83e5b7c2-9f8b-429f-980b-15fb7cbf3248 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031663 Aug 24 10:38:11.802914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1579/3024] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:11.854727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:11.855268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:11.862379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:11.862863 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:11.863301 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:11.863982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:11.864468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:11.864894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:11.926191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:11.934361 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-116ccd0f-9a6e-46e1-a99d-bd6ebeeca0e8 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 200 len: 1622 microversion: 2.1 time: 0.083567 Aug 24 10:38:11.936370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1446/3025] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:11 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 1622 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:11.953732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-bbd519bb-712e-4853-a233-54d938e50e1c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:11.954437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-bbd519bb-712e-4853-a233-54d938e50e1c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:12.011279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-bbd519bb-712e-4853-a233-54d938e50e1c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:12.018123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-bbd519bb-712e-4853-a233-54d938e50e1c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1237 microversion: 2.1 time: 0.068558 Aug 24 10:38:12.018682 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1580/3026] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:11 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1237 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:12.955811 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:12.956479 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: bdb467be-865b-489c-ab6c-f714a02672e1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:12.963461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:12.963943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:12.964413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:12.965102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:12.965626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:12.966051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:12.980516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Instance bdb467be-865b-489c-ab6c-f714a02672e1 could not be found. Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 HTTP/1.1 Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:38:12.982221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance bdb467be-865b-489c-ab6c-f714a02672e1 could not be found. Aug 24 10:38:12.984183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': 'bdb467be-865b-489c-ab6c-f714a02672e1'} Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bdb467be-865b-489c-ab6c-f714a02672e1 could not be found. Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:38:12.985536 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bdb467be-865b-489c-ab6c-f714a02672e1 could not be found. Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bdb467be-865b-489c-ab6c-f714a02672e1 could not be found. Aug 24 10:38:12.987627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:12.989150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 404 to user: Instance bdb467be-865b-489c-ab6c-f714a02672e1 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:38:12.989150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-277add15-30bc-4247-b777-052cec97c7c3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1" status: 404 len: 111 microversion: 2.1 time: 0.031686 Aug 24 10:38:12.989150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1447/3027] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:12 2022] GET /compute/v2.1/servers/bdb467be-865b-489c-ab6c-f714a02672e1 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:38:12.993044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-989e75dc-6353-46e5-b8b2-3c2cadd0b0ed tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '.version_select at 0x7fa33d7c7550>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:13.013914 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-989e75dc-6353-46e5-b8b2-3c2cadd0b0ed tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1354646386" status: 202 len: 0 microversion: 2.1 time: 0.025088 Aug 24 10:38:13.014495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1581/3028] 10.222.0.86 () {58 vars in 1221 bytes} [Wed Aug 24 10:38:12 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1354646386 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:38:13.039694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e3bb7ec6-0519-4b3c-96a4-242a5e5fb1f5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:13.040486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e3bb7ec6-0519-4b3c-96a4-242a5e5fb1f5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:13.102761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e3bb7ec6-0519-4b3c-96a4-242a5e5fb1f5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:13.109661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e3bb7ec6-0519-4b3c-96a4-242a5e5fb1f5 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1237 microversion: 2.1 time: 0.074810 Aug 24 10:38:13.110493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1448/3029] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:13 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1237 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:14.134244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a73e55df-1c19-482d-979c-fcfe464a43b7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:14.135295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a73e55df-1c19-482d-979c-fcfe464a43b7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:14.193155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a73e55df-1c19-482d-979c-fcfe464a43b7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:14.200159 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a73e55df-1c19-482d-979c-fcfe464a43b7 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1237 microversion: 2.1 time: 0.073317 Aug 24 10:38:14.200730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1582/3030] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:14 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1237 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:15.221366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:15.221980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:15.229525 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:15.230009 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:15.230445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:15.231129 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:15.231582 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:15.232008 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:15.292698 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:15.300454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d3bd3790-903d-4f44-905c-39b92a97f918 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1297 microversion: 2.1 time: 0.083111 Aug 24 10:38:15.301019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1449/3031] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:15 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1297 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:16.322600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:16.323168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:16.329765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:16.330240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:16.330672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:16.331351 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:16.331802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:16.332249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:16.392485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:16.400861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-264db7c2-18e0-410c-aa21-7949fdd3e837 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1297 microversion: 2.1 time: 0.082693 Aug 24 10:38:16.401427 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1583/3032] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:16 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1297 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:17.422234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:17.422815 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:17.429439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:17.429920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:17.430357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:17.431044 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:17.431499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:17.431924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:17.517153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:17.522037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-25133fc8-8527-49c4-ab8b-b73db688b613 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1297 microversion: 2.1 time: 0.103707 Aug 24 10:38:17.522965 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1450/3033] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:17 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1297 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:18.143463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:18.143967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:18.151224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:18.151745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:18.152304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:18.152912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:18.153464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:18.153780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:18.188614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:38:18.274077 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-be5fe2f7-7115-4a0c-ad42-41c4dde78ee3 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 204 len: 0 microversion: 2.1 time: 0.134271 Aug 24 10:38:18.274727 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1584/3034] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:38:18 2022] DELETE /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:38:18.286257 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:18.286967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:18.294366 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:18.294857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:18.295291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:18.295987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:18.296471 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:18.296893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:18.435263 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:18.443106 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4c3384b4-91e6-4ba9-a1b6-9bec6e60bdea tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1508 microversion: 2.1 time: 0.162876 Aug 24 10:38:18.443812 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1451/3035] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:18 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1508 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:38:18.545137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:18.545854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:18.551560 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:18.552060 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:18.552508 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:18.553188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:18.553644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:18.554068 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:18.761951 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1401291996", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "networks": [{"uuid": "7ce1e2bd-4840-4443-a463-bed805b1bf48"}]}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:18.820644 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] validate_networks() for [('7ce1e2bd-4840-4443-a463-bed805b1bf48', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:38:19.068561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:19.079180 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-afcfde4c-680f-4810-816e-e65f28f7926d tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1362 microversion: 2.1 time: 0.538118 Aug 24 10:38:19.079756 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1585/3036] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:18 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1362 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:19.237690 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:38:19.238351 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:38:19.238909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:38:19.239416 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:38:19.240247 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:38:19.249414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:19.249917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:19.250341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:19.251049 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:19.251492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:19.251906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:19.269341 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Getting quotas for project f1eac6259f65441d9abbabe2a04afad1. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:38:19.273568 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Getting quotas for user 4ce9eb7417a7482cb9cdaed16cb6aeb9 and project f1eac6259f65441d9abbabe2a04afad1. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:38:19.285458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:38:19.288034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:38:19.342226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-ea2c22e7-3149-4e97-8d71-1d8de860ecb7 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.584129 Aug 24 10:38:19.349002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1452/3037] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:38:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 586 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 24 10:38:19.354609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6b838a92-a2fe-4d72-bd27-62130ae62be4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:19.355226 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6b838a92-a2fe-4d72-bd27-62130ae62be4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:19.363703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-cd7b9bf6-c977-461c-9a7a-654ebc793dc7 req-763fdbe7-8325-4bf8-988d-d76ce3643714 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "efb6e5c0-4ef2-41af-8d0b-a410db398d96", "tag": "81225987-9fa1-4436-8538-21905fdd4e61"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:19.383410 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-cd7b9bf6-c977-461c-9a7a-654ebc793dc7 req-763fdbe7-8325-4bf8-988d-d76ce3643714 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:19.387872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-cd7b9bf6-c977-461c-9a7a-654ebc793dc7 req-763fdbe7-8325-4bf8-988d-d76ce3643714 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:19.388888 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-cd7b9bf6-c977-461c-9a7a-654ebc793dc7 req-763fdbe7-8325-4bf8-988d-d76ce3643714 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:19.393096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=100 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:19.402796 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-cd7b9bf6-c977-461c-9a7a-654ebc793dc7 req-763fdbe7-8325-4bf8-988d-d76ce3643714 service nova] Creating event network-changed:81225987-9fa1-4436-8538-21905fdd4e61 for instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 on hv-cinder-85427 Aug 24 10:38:19.408182 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-cd7b9bf6-c977-461c-9a7a-654ebc793dc7 req-763fdbe7-8325-4bf8-988d-d76ce3643714 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049317 Aug 24 10:38:19.409404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1453/3038] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:19.466832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:19.468318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:19.481208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:19.481816 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:19.482549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:19.483100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:19.483548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:19.483967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:19.511072 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6b838a92-a2fe-4d72-bd27-62130ae62be4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:19.524763 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6b838a92-a2fe-4d72-bd27-62130ae62be4 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1219 microversion: 2.1 time: 0.174062 Aug 24 10:38:19.537408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1586/3039] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:19 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1219 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:19.621772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:19.629711 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d601d14b-1784-4420-8afa-8c0a403c7da9 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1508 microversion: 2.1 time: 0.167717 Aug 24 10:38:19.630837 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1454/3040] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:19 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1508 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:38:20.101249 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:20.101899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:20.109034 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:20.109773 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:20.110323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:20.111062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:20.111549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:20.112134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:20.250906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:20.258485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a9dd27f8-e8a8-490e-9536-7ce96ad03769 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1362 microversion: 2.1 time: 0.161571 Aug 24 10:38:20.259705 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1587/3041] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:20 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1362 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:20.548135 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:20.548687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:20.555060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:20.555532 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:20.556625 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:20.557314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:20.557764 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:20.558184 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:20.618125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:20.627152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-751a6f02-8559-4e9a-8a2d-c5043a3df684 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1271 microversion: 2.1 time: 0.082996 Aug 24 10:38:20.627715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1455/3042] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:20 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1271 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:20.650601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:20.651222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:20.653790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=93,nova_api:DELETE=2,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:20.659320 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:20.659826 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:20.660277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:20.660971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:20.661455 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:20.661885 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:20.695019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:20.695495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:20.695929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:20.696401 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lazy-loading 'fault' on Instance uuid d6a8cae5-8006-46a9-90b3-4556894041b0 {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:38:20.736399 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:20.744433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8920a335-f999-4077-bfb1-b06889d6f324 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 200 len: 1312 microversion: 2.1 time: 0.097642 Aug 24 10:38:20.745011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1588/3043] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:20 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 1312 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:21.281943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:21.282552 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:21.288500 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.288982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.289415 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.290089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.290538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.290958 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.431403 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-7aa8ed38-8ad4-4630-8d48-8aac68f8165f req-06ae20c4-637d-46e6-9cb3-1caacbeb49e8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d6a8cae5-8006-46a9-90b3-4556894041b0", "name": "network-vif-deleted", "tag": "5930ef7d-e5d9-40c3-bc98-18c0ec1f1de4"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:21.436518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:21.446101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:21.453820 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-7aa8ed38-8ad4-4630-8d48-8aac68f8165f req-06ae20c4-637d-46e6-9cb3-1caacbeb49e8 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.456617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-7aa8ed38-8ad4-4630-8d48-8aac68f8165f req-06ae20c4-637d-46e6-9cb3-1caacbeb49e8 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.457103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-7aa8ed38-8ad4-4630-8d48-8aac68f8165f req-06ae20c4-637d-46e6-9cb3-1caacbeb49e8 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.466078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1f2b0da0-d260-46b8-ad89-204db1775fa0 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1362 microversion: 2.1 time: 0.187928 Aug 24 10:38:21.467909 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1456/3044] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:21 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1362 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:21.480937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-7aa8ed38-8ad4-4630-8d48-8aac68f8165f req-06ae20c4-637d-46e6-9cb3-1caacbeb49e8 service nova] Creating event network-vif-deleted:5930ef7d-e5d9-40c3-bc98-18c0ec1f1de4 for instance d6a8cae5-8006-46a9-90b3-4556894041b0 on hv-cinder-85427 Aug 24 10:38:21.498912 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-7aa8ed38-8ad4-4630-8d48-8aac68f8165f req-06ae20c4-637d-46e6-9cb3-1caacbeb49e8 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071623 Aug 24 10:38:21.499867 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1589/3045] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:38:21.650004 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:21.651457 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:21.658548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.659057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.659494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.660220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.660674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.661097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.725348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:21.733052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f988af2b-9bc1-4400-a6ae-8affc4a61c07 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1277 microversion: 2.1 time: 0.088830 Aug 24 10:38:21.733620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1457/3046] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:21 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1277 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:21.766422 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:21.767125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: d6a8cae5-8006-46a9-90b3-4556894041b0] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:21.773289 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.773768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.774198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.774913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.775365 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.775793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.789331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Instance d6a8cae5-8006-46a9-90b3-4556894041b0 could not be found. Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 HTTP/1.1 Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:38:21.791394 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance d6a8cae5-8006-46a9-90b3-4556894041b0 could not be found. Aug 24 10:38:21.793049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': 'd6a8cae5-8006-46a9-90b3-4556894041b0'} Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d6a8cae5-8006-46a9-90b3-4556894041b0 could not be found. Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:38:21.793968 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d6a8cae5-8006-46a9-90b3-4556894041b0 could not be found. Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d6a8cae5-8006-46a9-90b3-4556894041b0 could not be found. Aug 24 10:38:21.795575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:38:21.798235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 404 to user: Instance d6a8cae5-8006-46a9-90b3-4556894041b0 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:38:21.798235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9bd42916-c302-4ec3-b122-2676c7a6d896 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0" status: 404 len: 111 microversion: 2.1 time: 0.031143 Aug 24 10:38:21.798235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1590/3047] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:21 2022] GET /compute/v2.1/servers/d6a8cae5-8006-46a9-90b3-4556894041b0 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:38:21.801729 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:21.802223 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:21.808907 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.809400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.809833 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.810521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.810968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.811385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.839684 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:38:21.919179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cbdf1e54-c2d0-4ea6-832e-4589d4e4c58d tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "DELETE /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 204 len: 0 microversion: 2.1 time: 0.121470 Aug 24 10:38:21.919908 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1458/3048] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:38:21 2022] DELETE /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:38:21.929536 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:21.930191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:21.937701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.938190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.938630 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:21.939323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:21.939778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:21.940314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:22.231940 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:22.241247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9cdf526b-999c-47c1-8876-f970561bc6ad tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1521 microversion: 2.1 time: 0.316965 Aug 24 10:38:22.256309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1591/3049] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:21 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1521 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:22.489740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:22.490445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:22.492441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:22.498233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:22.498727 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:22.499162 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:22.499855 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:22.500333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:22.500760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:22.643574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:22.653933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-77e5e0e4-592f-4555-89e6-a0913bc506f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1362 microversion: 2.1 time: 0.168509 Aug 24 10:38:22.655172 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1459/3050] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:22 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1362 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:22.754271 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:22.754975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:22.761561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:22.762050 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:22.762491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:22.763186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:22.763648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:22.764075 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:23.084102 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:23.092962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7d99251f-cd76-4ec0-8c60-8c183ba569a2 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1319 microversion: 2.1 time: 0.342522 Aug 24 10:38:23.093540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1592/3051] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:22 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1319 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:23.263227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:23.263976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:23.270303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:23.270780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:23.271603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:23.272407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:23.272861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:23.273281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:23.344385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '46c01f8b99e24be5b6c2d73b9b1bd9b8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bdfd8ab42f9496d87a8cfcf78d7edc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:23.353828 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-103016d9-5d66-49f2-8fd4-ccd16b29a1b0 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 200 len: 1479 microversion: 2.1 time: 0.094775 Aug 24 10:38:23.365079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1460/3052] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:23 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 1479 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:23.526062 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-a310392f-3275-4757-adf0-4d08bf92f31d req-208f5da7-2a16-41f9-a67d-64f89ee123b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ab73a23-282f-4f54-9a62-5a417d7e7603", "name": "network-vif-deleted", "tag": "e2175a9a-3243-44ea-896c-f63d59862a02"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:23.535931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a310392f-3275-4757-adf0-4d08bf92f31d req-208f5da7-2a16-41f9-a67d-64f89ee123b8 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:23.536450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a310392f-3275-4757-adf0-4d08bf92f31d req-208f5da7-2a16-41f9-a67d-64f89ee123b8 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:23.536954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a310392f-3275-4757-adf0-4d08bf92f31d req-208f5da7-2a16-41f9-a67d-64f89ee123b8 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:23.549656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-a310392f-3275-4757-adf0-4d08bf92f31d req-208f5da7-2a16-41f9-a67d-64f89ee123b8 service nova] Creating event network-vif-deleted:e2175a9a-3243-44ea-896c-f63d59862a02 for instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 on hv-cinder-85427 Aug 24 10:38:23.555120 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-a310392f-3275-4757-adf0-4d08bf92f31d req-208f5da7-2a16-41f9-a67d-64f89ee123b8 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032853 Aug 24 10:38:23.555680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1593/3053] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:23.677509 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:23.678206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:23.684853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:23.685623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:23.686064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:23.686750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:23.687200 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:23.687620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:23.851683 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:23.862326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-97723baf-cb91-4973-92e8-3cc6dbaf2351 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1362 microversion: 2.1 time: 0.188852 Aug 24 10:38:23.863244 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1461/3054] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:23 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1362 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:38:24.115044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:24.115626 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:24.121829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:24.122325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:24.122768 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:24.123464 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:24.124146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:24.124581 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:24.292398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-6a4d828c-3023-4cfe-a643-d0f06d7ddb81 req-07af5b58-faf3-4216-8829-71be33732153 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "efb6e5c0-4ef2-41af-8d0b-a410db398d96", "name": "network-vif-plugged", "status": "completed", "tag": "81225987-9fa1-4436-8538-21905fdd4e61"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:24.302756 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-6a4d828c-3023-4cfe-a643-d0f06d7ddb81 req-07af5b58-faf3-4216-8829-71be33732153 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:24.303759 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-6a4d828c-3023-4cfe-a643-d0f06d7ddb81 req-07af5b58-faf3-4216-8829-71be33732153 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:24.304258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-6a4d828c-3023-4cfe-a643-d0f06d7ddb81 req-07af5b58-faf3-4216-8829-71be33732153 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:24.318976 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-6a4d828c-3023-4cfe-a643-d0f06d7ddb81 req-07af5b58-faf3-4216-8829-71be33732153 service nova] Creating event network-vif-plugged:81225987-9fa1-4436-8538-21905fdd4e61 for instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 on hv-cinder-85427 Aug 24 10:38:24.325326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-6a4d828c-3023-4cfe-a643-d0f06d7ddb81 req-07af5b58-faf3-4216-8829-71be33732153 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036820 Aug 24 10:38:24.326631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1462/3055] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:38:24.376574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:24.377576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] [instance: 7ab73a23-282f-4f54-9a62-5a417d7e7603] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:24.385484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:24.386191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:24.386879 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:24.387621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:24.388098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:24.388527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:24.405368 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] HTTP exception thrown: Instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 could not be found. Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Request method failure captured: Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 HTTP/1.1 Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdcb6wAvgSj-z0ZqXXODGFKw6oAI2O4LT1hx0943gqcV1-3Yg314rjj4w2kb9k8hF-1Ram54RnSX2qbjQqjha4GlJ77mXRMdMet1uzrHkZidxRsXdR_eW0SAs3koTZ2-F9SmI9s31ufkZgssy11Ojm1OA__NJXdqPmkw_wPqtzyEGU Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:38:24.407058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/1bdfd8ab42f9496d87a8cfcf78d7edc3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 1bdfd8ab42f9496d87a8cfcf78d7edc3 Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1910690561 Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 46c01f8b99e24be5b6c2d73b9b1bd9b8 Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1910690561-project Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 could not be found. Aug 24 10:38:24.409168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '7ab73a23-282f-4f54-9a62-5a417d7e7603'} Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 could not be found. Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:38:24.410046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 could not be found. Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 could not be found. Aug 24 10:38:24.411598 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:24.414230 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] Returning 404 to user: Instance 7ab73a23-282f-4f54-9a62-5a417d7e7603 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:38:24.415140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-06d7569d-2d11-4dc5-a797-9b2488ecfa47 tempest-ServerRescueNegativeTestJSON-1910690561 tempest-ServerRescueNegativeTestJSON-1910690561-project] 10.222.0.86 "GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603" status: 404 len: 111 microversion: 2.1 time: 0.043911 Aug 24 10:38:24.415774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1463/3056] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:24 2022] GET /compute/v2.1/servers/7ab73a23-282f-4f54-9a62-5a417d7e7603 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:38:24.627454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:24.638079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1fbe673f-5cd6-439e-86d8-02e45529bd8c tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1319 microversion: 2.1 time: 0.527848 Aug 24 10:38:24.638737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1594/3057] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:24 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1319 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:24.882986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:24.883584 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:24.890621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:24.891150 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:24.891593 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:24.892334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:24.892810 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:24.893413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:25.110268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:25.118372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8a0f32b1-f138-44f4-83ef-34116a7e39e6 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1362 microversion: 2.1 time: 0.239377 Aug 24 10:38:25.131967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1464/3058] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:24 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1362 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:25.576580 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-901b2c2b-5a82-4a35-98a7-6ceef4b81794 req-4eb0f69f-d68a-4ecc-bd30-3df63c47fe64 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7414457f-59d2-42da-8842-6d32e9898e89", "tag": "ca7423b9-1d89-4a56-a6c1-351cad10ad15"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:25.586233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-901b2c2b-5a82-4a35-98a7-6ceef4b81794 req-4eb0f69f-d68a-4ecc-bd30-3df63c47fe64 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:25.586713 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-901b2c2b-5a82-4a35-98a7-6ceef4b81794 req-4eb0f69f-d68a-4ecc-bd30-3df63c47fe64 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:25.587153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-901b2c2b-5a82-4a35-98a7-6ceef4b81794 req-4eb0f69f-d68a-4ecc-bd30-3df63c47fe64 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:25.598884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-901b2c2b-5a82-4a35-98a7-6ceef4b81794 req-4eb0f69f-d68a-4ecc-bd30-3df63c47fe64 service nova] Creating event network-changed:ca7423b9-1d89-4a56-a6c1-351cad10ad15 for instance 7414457f-59d2-42da-8842-6d32e9898e89 on hv-cinder-85427 Aug 24 10:38:25.604921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-901b2c2b-5a82-4a35-98a7-6ceef4b81794 req-4eb0f69f-d68a-4ecc-bd30-3df63c47fe64 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032264 Aug 24 10:38:25.605887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1595/3059] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:25.659448 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:25.662377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:25.669159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:25.669825 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:25.670427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:25.671160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:25.671617 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:25.672078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:25.944501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:25.954985 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0160ee38-6b0b-461f-b7d4-dcd3d58d0958 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1319 microversion: 2.1 time: 0.299836 Aug 24 10:38:25.955658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1465/3060] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:25 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1319 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:26.140004 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:26.140834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:26.147534 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:26.148393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:26.149284 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:26.150741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:26.151482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:26.152007 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:26.373152 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:26.385797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-10a53bce-6d54-413e-95a9-aa13be7c0816 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1362 microversion: 2.1 time: 0.249727 Aug 24 10:38:26.387092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1596/3061] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:26 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1362 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:26.975977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:26.976682 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:27.004889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:27.005700 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:27.007633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:27.015359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:27.016252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:27.017030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:27.266208 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:27.273905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-90eb6bce-1a88-41af-8a08-5026a9d0eecb tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1319 microversion: 2.1 time: 0.301951 Aug 24 10:38:27.274480 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1466/3062] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:26 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1319 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:38:27.407933 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:27.409406 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:27.417631 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:27.420151 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:27.420806 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:27.421761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:27.422432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:27.423094 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:27.677263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:27.685518 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6ac18dab-fb8c-47e3-972d-ca7f62602b73 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1549 microversion: 2.1 time: 0.281633 Aug 24 10:38:27.686092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1597/3063] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:27 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1549 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:27.709864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7b1eff17-de7e-4e2c-aa28-c04f3ffb67cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:27.710651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7b1eff17-de7e-4e2c-aa28-c04f3ffb67cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:27.717517 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b1eff17-de7e-4e2c-aa28-c04f3ffb67cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:27.718317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b1eff17-de7e-4e2c-aa28-c04f3ffb67cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:27.718997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b1eff17-de7e-4e2c-aa28-c04f3ffb67cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:27.719932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b1eff17-de7e-4e2c-aa28-c04f3ffb67cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:27.720659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b1eff17-de7e-4e2c-aa28-c04f3ffb67cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:27.721314 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b1eff17-de7e-4e2c-aa28-c04f3ffb67cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:27.815051 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7b1eff17-de7e-4e2c-aa28-c04f3ffb67cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.109012 Aug 24 10:38:27.815662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1467/3064] 10.222.0.86 () {58 vars in 1275 bytes} [Wed Aug 24 10:38:27 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-interface => generated 285 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:28.296457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:28.297101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:28.309922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:28.310421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:28.310859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:28.311755 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:28.312246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:28.312673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:28.348231 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-f29bcb15-a6ee-40bc-992e-adea133efc0b req-0889258a-e9cc-41e0-9a22-7d4de043d12f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7414457f-59d2-42da-8842-6d32e9898e89", "name": "network-vif-plugged", "status": "completed", "tag": "ca7423b9-1d89-4a56-a6c1-351cad10ad15"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:28.360763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f29bcb15-a6ee-40bc-992e-adea133efc0b req-0889258a-e9cc-41e0-9a22-7d4de043d12f service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:28.361501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f29bcb15-a6ee-40bc-992e-adea133efc0b req-0889258a-e9cc-41e0-9a22-7d4de043d12f service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:28.362179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f29bcb15-a6ee-40bc-992e-adea133efc0b req-0889258a-e9cc-41e0-9a22-7d4de043d12f service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:28.380921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-f29bcb15-a6ee-40bc-992e-adea133efc0b req-0889258a-e9cc-41e0-9a22-7d4de043d12f service nova] Creating event network-vif-plugged:ca7423b9-1d89-4a56-a6c1-351cad10ad15 for instance 7414457f-59d2-42da-8842-6d32e9898e89 on hv-cinder-85427 Aug 24 10:38:28.386659 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-f29bcb15-a6ee-40bc-992e-adea133efc0b req-0889258a-e9cc-41e0-9a22-7d4de043d12f service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042868 Aug 24 10:38:28.387781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1468/3065] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:38:28.538229 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:28.548145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9dabbd53-f27d-4128-94bc-5173d14a6149 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1319 microversion: 2.1 time: 0.257176 Aug 24 10:38:28.549600 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1598/3066] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:28 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1319 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:29.187620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-124e4823-c326-4e1b-8da7-acb63ad9a125 req-a7951392-3a66-4098-a756-f24ed7c9b0aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "efb6e5c0-4ef2-41af-8d0b-a410db398d96", "tag": "81225987-9fa1-4436-8538-21905fdd4e61"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:29.197803 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-124e4823-c326-4e1b-8da7-acb63ad9a125 req-a7951392-3a66-4098-a756-f24ed7c9b0aa service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:29.198285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-124e4823-c326-4e1b-8da7-acb63ad9a125 req-a7951392-3a66-4098-a756-f24ed7c9b0aa service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:29.198738 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-124e4823-c326-4e1b-8da7-acb63ad9a125 req-a7951392-3a66-4098-a756-f24ed7c9b0aa service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:29.209798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-124e4823-c326-4e1b-8da7-acb63ad9a125 req-a7951392-3a66-4098-a756-f24ed7c9b0aa service nova] Creating event network-changed:81225987-9fa1-4436-8538-21905fdd4e61 for instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 on hv-cinder-85427 Aug 24 10:38:29.215048 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-124e4823-c326-4e1b-8da7-acb63ad9a125 req-a7951392-3a66-4098-a756-f24ed7c9b0aa service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.241836 Aug 24 10:38:29.215642 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1469/3067] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:29.571147 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:29.573665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:29.577358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=16 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:29.581920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:29.582335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:29.582945 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:29.583828 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:29.584491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:29.585121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:29.606894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:29.761973 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:29.786087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1ff6894b-d470-45a9-b337-2bd71545d0bd tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1319 microversion: 2.1 time: 0.220695 Aug 24 10:38:29.787241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1599/3068] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:29 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1319 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:30.807970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:30.808774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=10 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:30.813392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:30.814086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:30.821139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:30.821613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:30.822040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:30.823424 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:30.823877 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:30.824325 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:30.920849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-89935584-9035-48d1-a60e-f140aac80613 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa33d816160>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:30.971108 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-89935584-9035-48d1-a60e-f140aac80613 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/6950c3f6-e121-4f21-8455-a88d26f626a3" status: 200 len: 350 microversion: 2.1 time: 0.054203 Aug 24 10:38:30.971683 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1600/3069] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:38:30 2022] GET /compute/v2.1/os-volumes/6950c3f6-e121-4f21-8455-a88d26f626a3 => generated 350 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:38:31.230183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:31.249510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cc7220ad-ac52-4be6-9c70-cdccf0356082 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1508 microversion: 2.1 time: 0.442625 Aug 24 10:38:31.251307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1470/3070] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:30 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1508 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:31.277058 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c2d3416a-167d-42df-91df-a79d15faef97 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "536d5379-eab3-4d25-9d7a-d986f1aeb297", "device": "/dev/sdb"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:31.282646 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c2d3416a-167d-42df-91df-a79d15faef97 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:31.297253 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d3416a-167d-42df-91df-a79d15faef97 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:31.298057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d3416a-167d-42df-91df-a79d15faef97 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:31.298841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d3416a-167d-42df-91df-a79d15faef97 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:31.300100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d3416a-167d-42df-91df-a79d15faef97 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:31.300990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d3416a-167d-42df-91df-a79d15faef97 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:31.301708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d3416a-167d-42df-91df-a79d15faef97 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:31.558237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c2d3416a-167d-42df-91df-a79d15faef97 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "POST /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.290210 Aug 24 10:38:31.558749 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1601/3071] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:38:31 2022] POST /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89/os-volume_attachments => generated 194 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:32.315791 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8db04cd5-5e4f-406c-9f44-53c3cde12111 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa33d8501f0>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:32.364520 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8db04cd5-5e4f-406c-9f44-53c3cde12111 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/7609d8ae-40c0-47ff-8898-fdf4a9ddced6" status: 200 len: 351 microversion: 2.1 time: 0.052555 Aug 24 10:38:32.365108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1471/3072] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:38:32 2022] GET /compute/v2.1/os-volumes/7609d8ae-40c0-47ff-8898-fdf4a9ddced6 => generated 351 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:33.696415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5c5de3ec-e8dd-4b77-954a-82a6a04944ab tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa33d9e6550>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:33.750121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5c5de3ec-e8dd-4b77-954a-82a6a04944ab tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/be08c451-877b-4b81-a9a8-f63c3f19d7c2" status: 200 len: 350 microversion: 2.1 time: 0.058108 Aug 24 10:38:33.750687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1602/3073] 10.222.0.86 () {58 vars in 1245 bytes} [Wed Aug 24 10:38:33 2022] GET /compute/v2.1/os-volumes/be08c451-877b-4b81-a9a8-f63c3f19d7c2 => generated 350 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:33.764100 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-61ab0b8d-0779-4132-90a0-8ee21d84650b tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa342252280>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:33.844490 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-61ab0b8d-0779-4132-90a0-8ee21d84650b tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.085270 Aug 24 10:38:33.845055 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1472/3074] 10.222.0.86 () {58 vars in 1134 bytes} [Wed Aug 24 10:38:33 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:33.858869 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-08459718-3895-4989-9c9a-740d659d56ea tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa33d816550>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:33.941219 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-08459718-3895-4989-9c9a-740d659d56ea tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.086601 Aug 24 10:38:33.941785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1603/3075] 10.222.0.86 () {58 vars in 1149 bytes} [Wed Aug 24 10:38:33 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:33.955285 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b50b3643-2320-4572-8327-c4a790471fa9 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa3401d8c10>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:34.036707 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b50b3643-2320-4572-8327-c4a790471fa9 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.085142 Aug 24 10:38:34.037274 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1473/3076] 10.222.0.86 () {58 vars in 1134 bytes} [Wed Aug 24 10:38:33 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:34.050132 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c1f07ebc-0ab1-4728-a581-6f9bf3cab677 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa33d960dc0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:34.132896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c1f07ebc-0ab1-4728-a581-6f9bf3cab677 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.086476 Aug 24 10:38:34.133457 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1604/3077] 10.222.0.86 () {58 vars in 1167 bytes} [Wed Aug 24 10:38:34 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:34.146152 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-736ddd6a-aeb1-4b8b-903d-813d95b6ee57 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa33d8c4820>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:34.229823 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-736ddd6a-aeb1-4b8b-903d-813d95b6ee57 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.087327 Aug 24 10:38:34.230393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1474/3078] 10.222.0.86 () {58 vars in 1170 bytes} [Wed Aug 24 10:38:34 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:34.243314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6f902651-5c20-4336-82dc-887dc4ec7352 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa33d816280>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:34.324231 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6f902651-5c20-4336-82dc-887dc4ec7352 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.084561 Aug 24 10:38:34.324796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1605/3079] 10.222.0.86 () {58 vars in 1155 bytes} [Wed Aug 24 10:38:34 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:34.337585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-90d1e6e5-25ef-48d4-91d5-45cfb104b536 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa342252af0>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:34.423157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-90d1e6e5-25ef-48d4-91d5-45cfb104b536 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.089288 Aug 24 10:38:34.423783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1475/3080] 10.222.0.86 () {58 vars in 1188 bytes} [Wed Aug 24 10:38:34 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:38:34.436318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0b74efd4-6d3d-44b9-8226-c5d0faad1858 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] Calling method '.version_select at 0x7fa33d89daf0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:34.518006 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0b74efd4-6d3d-44b9-8226-c5d0faad1858 tempest-VolumesTestJSON-834481968 tempest-VolumesTestJSON-834481968-project] 10.222.0.86 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.085314 Aug 24 10:38:34.518649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1606/3081] 10.222.0.86 () {58 vars in 1155 bytes} [Wed Aug 24 10:38:34 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:51.985374 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=23 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:51.986236 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:51.990600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:51.991115 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:51.999491 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:51.999970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:52.000430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:52.001109 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:52.001553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:52.001970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:52.087817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lazy-loading 'services' on Instance uuid 7414457f-59d2-42da-8842-6d32e9898e89 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:38:52.160612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6a2242b4-9461-434f-ad28-6da81746d239 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89/os-volume_attachments/536d5379-eab3-4d25-9d7a-d986f1aeb297" status: 202 len: 0 microversion: 2.1 time: 0.176243 Aug 24 10:38:52.161264 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1476/3082] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:38:51 2022] DELETE /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89/os-volume_attachments/536d5379-eab3-4d25-9d7a-d986f1aeb297 => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 24 10:38:53.285795 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=27 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:53.286644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=13,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:38:53.290976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:53.291591 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:53.300495 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:53.300977 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:53.301609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:53.302302 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:53.302758 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:53.303188 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:53.328742 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:38:53.401674 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5ca89d3e-aafd-4bc1-83e5-140bfe4b6bf0 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "DELETE /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 204 len: 0 microversion: 2.1 time: 0.116821 Aug 24 10:38:53.405710 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1607/3083] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:38:53 2022] DELETE /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:38:53.414029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:53.414818 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:53.422013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:53.422496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:53.422929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:53.423620 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:53.424095 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:53.424524 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:53.555949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:53.563710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c91a8a5a-ca47-4683-88e1-ab499553f69f tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1514 microversion: 2.1 time: 0.156173 Aug 24 10:38:53.564297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1477/3084] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:53 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1514 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:54.585030 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:54.585736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:54.592018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:54.592525 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:54.592962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:54.593643 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:54.594098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:54.594540 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:54.727217 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:54.737040 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e68429b5-32e8-4312-8238-f48a6c146eff tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1514 microversion: 2.1 time: 0.155985 Aug 24 10:38:54.737876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1608/3085] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:54 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1514 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:55.768556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-37250caf-4d3b-4cff-96c8-3e885dc84ca0 req-5ae0640e-d739-4df6-a124-80664c70c048 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7414457f-59d2-42da-8842-6d32e9898e89", "name": "network-vif-deleted", "tag": "ca7423b9-1d89-4a56-a6c1-351cad10ad15"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:38:55.769746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:55.770442 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:55.794944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-37250caf-4d3b-4cff-96c8-3e885dc84ca0 req-5ae0640e-d739-4df6-a124-80664c70c048 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:55.795395 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-37250caf-4d3b-4cff-96c8-3e885dc84ca0 req-5ae0640e-d739-4df6-a124-80664c70c048 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:55.795660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:55.798079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:55.798531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:55.799073 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-37250caf-4d3b-4cff-96c8-3e885dc84ca0 req-5ae0640e-d739-4df6-a124-80664c70c048 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:55.801706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:55.802660 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:55.803103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:55.818553 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-37250caf-4d3b-4cff-96c8-3e885dc84ca0 req-5ae0640e-d739-4df6-a124-80664c70c048 service nova] Creating event network-vif-deleted:ca7423b9-1d89-4a56-a6c1-351cad10ad15 for instance 7414457f-59d2-42da-8842-6d32e9898e89 on hv-cinder-85427 Aug 24 10:38:55.824747 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-37250caf-4d3b-4cff-96c8-3e885dc84ca0 req-5ae0640e-d739-4df6-a124-80664c70c048 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069608 Aug 24 10:38:55.825955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1478/3086] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:38:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:38:55.887230 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ce9eb7417a7482cb9cdaed16cb6aeb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1eac6259f65441d9abbabe2a04afad1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:38:55.895466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-69dbb61d-6818-42ad-bd38-6c443f3e569a tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 200 len: 1304 microversion: 2.1 time: 0.132957 Aug 24 10:38:55.896343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1609/3087] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:55 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 1304 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:38:56.917540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:38:56.918281 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] [instance: 7414457f-59d2-42da-8842-6d32e9898e89] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:38:56.924608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:56.925093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:56.925533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:56.926218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:38:56.926673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:38:56.927097 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:38:56.940710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] HTTP exception thrown: Instance 7414457f-59d2-42da-8842-6d32e9898e89 could not be found. Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Request method failure captured: Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 HTTP/1.1 Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfgM_TxkhYU1OGucJX_PJYzYYB02cQoEuONc1lDzw6U2shvqqEOYn_FcXh9lg3o6iLu8YwoHnExpCQhJo9BHPGBS8Web1Gb1gq7knEkNJz-JQq9MVGzl-lz3hCtkQmVxiuli94lDPkkwFkd_-utKUjA5hP0x_ikwQiM_vWuHqR695Ew Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:38:56.942348 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/f1eac6259f65441d9abbabe2a04afad1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: f1eac6259f65441d9abbabe2a04afad1 Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1686923724 Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 4ce9eb7417a7482cb9cdaed16cb6aeb9 Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-VolumesSnapshotTestJSON-1686923724-project Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 7414457f-59d2-42da-8842-6d32e9898e89 could not be found. Aug 24 10:38:56.944237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '7414457f-59d2-42da-8842-6d32e9898e89'} Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7414457f-59d2-42da-8842-6d32e9898e89 could not be found. Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:38:56.945173 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7414457f-59d2-42da-8842-6d32e9898e89 could not be found. Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7414457f-59d2-42da-8842-6d32e9898e89 could not be found. Aug 24 10:38:56.946745 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:38:56.948817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] Returning 404 to user: Instance 7414457f-59d2-42da-8842-6d32e9898e89 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:38:56.948817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2c066de2-8677-47a7-8096-73f22868c729 tempest-VolumesSnapshotTestJSON-1686923724 tempest-VolumesSnapshotTestJSON-1686923724-project] 10.222.0.86 "GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89" status: 404 len: 111 microversion: 2.1 time: 0.030492 Aug 24 10:38:56.948817 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1479/3088] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:38:56 2022] GET /compute/v2.1/servers/7414457f-59d2-42da-8842-6d32e9898e89 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:39:13.641482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-47cc071a-84f7-445c-a05a-f497eca6f0cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:13.642134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-47cc071a-84f7-445c-a05a-f497eca6f0cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:13.644934 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:13.645688 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:13.651575 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47cc071a-84f7-445c-a05a-f497eca6f0cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:13.652088 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47cc071a-84f7-445c-a05a-f497eca6f0cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:13.652530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47cc071a-84f7-445c-a05a-f497eca6f0cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:13.653214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47cc071a-84f7-445c-a05a-f497eca6f0cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:13.653662 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47cc071a-84f7-445c-a05a-f497eca6f0cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:13.654084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-47cc071a-84f7-445c-a05a-f497eca6f0cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:13.679337 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-47cc071a-84f7-445c-a05a-f497eca6f0cb tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/ips" status: 200 len: 146 microversion: 2.1 time: 0.041708 Aug 24 10:39:13.679900 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1610/3089] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:39:13 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/ips => generated 146 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:15.014024 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c0a7e695-bb79-4519-a3d6-5a84b7a97551 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "95b08696-2e43-4869-a18f-f9e1b2b813b0"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:39:15.016376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c0a7e695-bb79-4519-a3d6-5a84b7a97551 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:15.018935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=8 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:15.019592 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=19,nova_cell1:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:15.029206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0a7e695-bb79-4519-a3d6-5a84b7a97551 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:15.029702 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0a7e695-bb79-4519-a3d6-5a84b7a97551 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:15.030142 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0a7e695-bb79-4519-a3d6-5a84b7a97551 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:15.030854 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0a7e695-bb79-4519-a3d6-5a84b7a97551 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:15.031601 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0a7e695-bb79-4519-a3d6-5a84b7a97551 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:15.032034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c0a7e695-bb79-4519-a3d6-5a84b7a97551 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:15.293391 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c0a7e695-bb79-4519-a3d6-5a84b7a97551 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.283382 Aug 24 10:39:15.294022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1480/3090] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:39:15 2022] POST /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments => generated 194 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:21.770793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0d7bfaca-8edd-4f38-b8bc-650989f72900 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:21.773321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0d7bfaca-8edd-4f38-b8bc-650989f72900 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:21.780208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d7bfaca-8edd-4f38-b8bc-650989f72900 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:21.780747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d7bfaca-8edd-4f38-b8bc-650989f72900 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:21.781177 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d7bfaca-8edd-4f38-b8bc-650989f72900 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:21.782012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d7bfaca-8edd-4f38-b8bc-650989f72900 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:21.782323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d7bfaca-8edd-4f38-b8bc-650989f72900 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:21.782753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0d7bfaca-8edd-4f38-b8bc-650989f72900 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:21.819745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0d7bfaca-8edd-4f38-b8bc-650989f72900 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054793 Aug 24 10:39:21.820333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1611/3091] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:39:21 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:21.832663 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d8fd7b12-c36d-4e52-ad56-1fc92e470230 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:21.833256 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d8fd7b12-c36d-4e52-ad56-1fc92e470230 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:21.839881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8fd7b12-c36d-4e52-ad56-1fc92e470230 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:21.840464 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8fd7b12-c36d-4e52-ad56-1fc92e470230 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:21.840852 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8fd7b12-c36d-4e52-ad56-1fc92e470230 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:21.841547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8fd7b12-c36d-4e52-ad56-1fc92e470230 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:21.842002 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8fd7b12-c36d-4e52-ad56-1fc92e470230 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:21.842426 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d8fd7b12-c36d-4e52-ad56-1fc92e470230 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:21.871514 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d8fd7b12-c36d-4e52-ad56-1fc92e470230 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/95b08696-2e43-4869-a18f-f9e1b2b813b0" status: 200 len: 194 microversion: 2.1 time: 0.044085 Aug 24 10:39:21.876101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1481/3092] 10.222.0.86 () {58 vars in 1413 bytes} [Wed Aug 24 10:39:21 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/95b08696-2e43-4869-a18f-f9e1b2b813b0 => generated 194 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:23.273896 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3273a249-85d2-4b6a-aa13-44bd645bc834 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6f60ca68-5b4b-453b-bf82-db1ed3e4d6c0"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:39:23.276328 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3273a249-85d2-4b6a-aa13-44bd645bc834 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:23.283003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3273a249-85d2-4b6a-aa13-44bd645bc834 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:23.283489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3273a249-85d2-4b6a-aa13-44bd645bc834 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:23.283927 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3273a249-85d2-4b6a-aa13-44bd645bc834 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:23.284655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3273a249-85d2-4b6a-aa13-44bd645bc834 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:23.285107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3273a249-85d2-4b6a-aa13-44bd645bc834 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:23.285785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3273a249-85d2-4b6a-aa13-44bd645bc834 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:23.541728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3273a249-85d2-4b6a-aa13-44bd645bc834 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "POST /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.275417 Aug 24 10:39:23.542551 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1612/3093] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:39:23 2022] POST /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments => generated 194 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:31.071499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3281cdff-eaf5-4d69-98d7-8f45b1d20172 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:31.074043 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3281cdff-eaf5-4d69-98d7-8f45b1d20172 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:31.080631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3281cdff-eaf5-4d69-98d7-8f45b1d20172 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:31.081116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3281cdff-eaf5-4d69-98d7-8f45b1d20172 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:31.081548 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3281cdff-eaf5-4d69-98d7-8f45b1d20172 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:31.082232 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3281cdff-eaf5-4d69-98d7-8f45b1d20172 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:31.082685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3281cdff-eaf5-4d69-98d7-8f45b1d20172 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:31.083114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3281cdff-eaf5-4d69-98d7-8f45b1d20172 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:31.113313 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3281cdff-eaf5-4d69-98d7-8f45b1d20172 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.045772 Aug 24 10:39:31.113856 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1482/3094] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:39:31 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments => generated 371 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:31.258611 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7af8e499-1551-4013-a96b-f559bbf68f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:31.259170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7af8e499-1551-4013-a96b-f559bbf68f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:31.265268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7af8e499-1551-4013-a96b-f559bbf68f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:31.265747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7af8e499-1551-4013-a96b-f559bbf68f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:31.266178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7af8e499-1551-4013-a96b-f559bbf68f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:31.266858 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7af8e499-1551-4013-a96b-f559bbf68f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:31.267309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7af8e499-1551-4013-a96b-f559bbf68f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:31.267731 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7af8e499-1551-4013-a96b-f559bbf68f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:31.296405 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7af8e499-1551-4013-a96b-f559bbf68f29 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/95b08696-2e43-4869-a18f-f9e1b2b813b0" status: 200 len: 194 microversion: 2.1 time: 0.043181 Aug 24 10:39:31.297208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1613/3095] 10.222.0.86 () {58 vars in 1413 bytes} [Wed Aug 24 10:39:31 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/95b08696-2e43-4869-a18f-f9e1b2b813b0 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:31.307101 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:31.307589 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:31.313427 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:31.313910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:31.314355 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:31.315037 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:31.315494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:31.315919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:31.396838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lazy-loading 'services' on Instance uuid efb6e5c0-4ef2-41af-8d0b-a410db398d96 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:39:31.480411 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-28c29613-9ccf-4549-a9ce-8ccb9be7d935 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "DELETE /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/95b08696-2e43-4869-a18f-f9e1b2b813b0" status: 202 len: 0 microversion: 2.1 time: 0.176807 Aug 24 10:39:31.481164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1483/3096] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:39:31 2022] DELETE /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/95b08696-2e43-4869-a18f-f9e1b2b813b0 => generated 0 bytes in 179 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:39:32.605692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-866ae0dc-c734-4330-a423-87584f00d996 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:32.606209 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-866ae0dc-c734-4330-a423-87584f00d996 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:32.612622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-866ae0dc-c734-4330-a423-87584f00d996 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:32.613107 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-866ae0dc-c734-4330-a423-87584f00d996 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:32.613549 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-866ae0dc-c734-4330-a423-87584f00d996 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:32.614235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-866ae0dc-c734-4330-a423-87584f00d996 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:32.614694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-866ae0dc-c734-4330-a423-87584f00d996 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:32.615123 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-866ae0dc-c734-4330-a423-87584f00d996 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:32.643557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-866ae0dc-c734-4330-a423-87584f00d996 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/6f60ca68-5b4b-453b-bf82-db1ed3e4d6c0" status: 200 len: 194 microversion: 2.1 time: 0.042351 Aug 24 10:39:32.644066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1614/3097] 10.222.0.86 () {58 vars in 1413 bytes} [Wed Aug 24 10:39:32 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/6f60ca68-5b4b-453b-bf82-db1ed3e4d6c0 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:32.655379 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:32.656038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:32.662923 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:32.663399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:32.663835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:32.664554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:32.665005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:32.665437 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:32.742992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lazy-loading 'services' on Instance uuid efb6e5c0-4ef2-41af-8d0b-a410db398d96 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:39:32.813005 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7b919c8f-7533-4cd2-b75b-9fd5731a53a3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "DELETE /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/6f60ca68-5b4b-453b-bf82-db1ed3e4d6c0" status: 202 len: 0 microversion: 2.1 time: 0.161723 Aug 24 10:39:32.813590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1484/3098] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:39:32 2022] DELETE /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments/6f60ca68-5b4b-453b-bf82-db1ed3e4d6c0 => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:39:33.980601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7735460a-f936-4034-8373-a3dda8e65675 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:33.982812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7735460a-f936-4034-8373-a3dda8e65675 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:33.989067 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7735460a-f936-4034-8373-a3dda8e65675 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:33.989578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7735460a-f936-4034-8373-a3dda8e65675 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:33.990016 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7735460a-f936-4034-8373-a3dda8e65675 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:33.990691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7735460a-f936-4034-8373-a3dda8e65675 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:33.991142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7735460a-f936-4034-8373-a3dda8e65675 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:33.991566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7735460a-f936-4034-8373-a3dda8e65675 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:34.021531 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7735460a-f936-4034-8373-a3dda8e65675 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045420 Aug 24 10:39:34.022317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1615/3099] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:39:33 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 24 10:39:34.136614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f37b4ef2-3321-4887-bfe0-bd7d58b2a598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:34.138866 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f37b4ef2-3321-4887-bfe0-bd7d58b2a598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:34.145081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f37b4ef2-3321-4887-bfe0-bd7d58b2a598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:34.145557 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f37b4ef2-3321-4887-bfe0-bd7d58b2a598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:34.145988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f37b4ef2-3321-4887-bfe0-bd7d58b2a598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:34.146687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f37b4ef2-3321-4887-bfe0-bd7d58b2a598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:34.147139 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f37b4ef2-3321-4887-bfe0-bd7d58b2a598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:34.147560 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f37b4ef2-3321-4887-bfe0-bd7d58b2a598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:34.177031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f37b4ef2-3321-4887-bfe0-bd7d58b2a598 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045659 Aug 24 10:39:34.177588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1485/3100] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:39:34 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 24 10:39:34.238220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:34.238717 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:34.244644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:34.245122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:34.245562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:34.246245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:34.246701 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:34.247128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:34.271418 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:39:34.345404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-fab8627e-db19-45e2-8a28-21abff7b0e74 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "DELETE /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 204 len: 0 microversion: 2.1 time: 0.110616 Aug 24 10:39:34.346258 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1616/3101] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:39:34 2022] DELETE /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:39:34.354400 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:34.355010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:34.361216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:34.361718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:34.362143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:34.362832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:34.363287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:34.363716 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:34.498565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:34.506521 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-9b7dde8f-c7e8-48b1-894f-f582d2ca2fb8 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1673 microversion: 2.1 time: 0.155977 Aug 24 10:39:34.507081 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1486/3102] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:34 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1673 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:39:35.529031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:35.529735 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:35.536126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:35.536883 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:35.537312 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:35.538000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:35.538461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:35.538893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:35.676137 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:35.684474 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7812c99f-d08d-49bf-a422-deff64ab9d25 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1673 microversion: 2.1 time: 0.159764 Aug 24 10:39:35.685033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1617/3103] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:35 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1673 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:39:36.706287 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:36.706906 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:36.714003 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:36.714701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:36.715136 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:36.715921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:36.716414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:36.716836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:36.855717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:36.866235 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-404f6118-73b0-48d2-a728-30f7241cef22 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1673 microversion: 2.1 time: 0.164382 Aug 24 10:39:36.867014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1487/3104] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:36 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1673 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:39:37.408680 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-0baef77e-c09f-4158-b769-03816a2c3fbf req-4da20ba0-12fd-4b81-b72c-b56d13229b43 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "efb6e5c0-4ef2-41af-8d0b-a410db398d96", "name": "network-vif-deleted", "tag": "81225987-9fa1-4436-8538-21905fdd4e61"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:39:37.419697 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-0baef77e-c09f-4158-b769-03816a2c3fbf req-4da20ba0-12fd-4b81-b72c-b56d13229b43 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:37.420234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-0baef77e-c09f-4158-b769-03816a2c3fbf req-4da20ba0-12fd-4b81-b72c-b56d13229b43 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:37.420800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-0baef77e-c09f-4158-b769-03816a2c3fbf req-4da20ba0-12fd-4b81-b72c-b56d13229b43 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:37.432982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-0baef77e-c09f-4158-b769-03816a2c3fbf req-4da20ba0-12fd-4b81-b72c-b56d13229b43 service nova] Creating event network-vif-deleted:81225987-9fa1-4436-8538-21905fdd4e61 for instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 on hv-cinder-85427 Aug 24 10:39:37.439191 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-0baef77e-c09f-4158-b769-03816a2c3fbf req-4da20ba0-12fd-4b81-b72c-b56d13229b43 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034081 Aug 24 10:39:37.440197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1618/3105] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:39:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:37.889067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:37.889924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:37.896715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:37.896978 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:37.897422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:37.898989 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:37.900170 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:37.900605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:37.939586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:37.940107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:37.940541 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:37.941761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.objects.instance [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lazy-loading 'fault' on Instance uuid efb6e5c0-4ef2-41af-8d0b-a410db398d96 {{(pid=120654) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:39:37.987579 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1de97cce55e944dbb62a920ea11734f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e055ba4967db42108c069ebc3843dcf6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:37.995376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b9a9a6c4-f5e2-4c1e-b519-4969157a36f9 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 200 len: 1327 microversion: 2.1 time: 0.110064 Aug 24 10:39:37.995897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1488/3106] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:37 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 1327 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:39:39.018055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:39.018752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:39.025918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:39.026599 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:39.027052 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:39.027790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:39.028285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:39.028719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:39.043045 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] HTTP exception thrown: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Request method failure captured: Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 HTTP/1.1 Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfei3ERPSR8lry9h62i-mTjfLEiY_G0pf4TOSE4I4STV_SQYl2_UIncspaIl58uBJJzOdoQ7pLY4XfcLnM6PIi-fCQvX7S1I2xM3rJol0zQr6Uy5AXr20ftWYLN4Ls8-en59yYA6dcGI9oXswbuIZaITfY674PQC2m8u_HkGCNmRiYI Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:39:39.044740 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 1de97cce55e944dbb62a920ea11734f5 Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:39.046649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': 'efb6e5c0-4ef2-41af-8d0b-a410db398d96'} Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:39:39.047696 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:39.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:39.051087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Returning 404 to user: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:39:39.051087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1fb92ebe-d9ce-4483-836b-05e7e2ec3f3b tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 404 len: 111 microversion: 2.1 time: 0.032843 Aug 24 10:39:39.051087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1619/3107] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:39 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:39:39.054896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7470c57e-87f6-4e84-8595-b9d71bfd0606 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '.version_select at 0x7fa342252940>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:39.074428 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7470c57e-87f6-4e84-8595-b9d71bfd0606 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1355527141" status: 202 len: 0 microversion: 2.1 time: 0.023257 Aug 24 10:39:39.075072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1489/3108] 10.222.0.86 () {58 vars in 1221 bytes} [Wed Aug 24 10:39:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1355527141 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:39:41.945562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:41.946058 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:41.952785 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:41.953372 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:41.953733 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:41.954437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:41.954898 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:41.955335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:41.971136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] HTTP exception thrown: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Request method failure captured: Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 HTTP/1.1 Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfei3ERPSR8lry9h62i-mTjfLEiY_G0pf4TOSE4I4STV_SQYl2_UIncspaIl58uBJJzOdoQ7pLY4XfcLnM6PIi-fCQvX7S1I2xM3rJol0zQr6Uy5AXr20ftWYLN4Ls8-en59yYA6dcGI9oXswbuIZaITfY674PQC2m8u_HkGCNmRiYI Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:39:41.972734 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 1de97cce55e944dbb62a920ea11734f5 Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:41.974703 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': 'efb6e5c0-4ef2-41af-8d0b-a410db398d96'} Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:39:41.975607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:41.977078 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:41.979183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Returning 404 to user: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:39:41.979183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-cbcfa9a7-25d2-4498-bb5f-220164026d15 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "DELETE /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 404 len: 111 microversion: 2.1 time: 0.032377 Aug 24 10:39:41.979183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1620/3109] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:39:41 2022] DELETE /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:39:41.983701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:41.984344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: efb6e5c0-4ef2-41af-8d0b-a410db398d96] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:41.990853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:41.991338 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:41.991777 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:41.992488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:41.992944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:41.993371 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:42.007204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] HTTP exception thrown: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Request method failure captured: Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 HTTP/1.1 Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfei3ERPSR8lry9h62i-mTjfLEiY_G0pf4TOSE4I4STV_SQYl2_UIncspaIl58uBJJzOdoQ7pLY4XfcLnM6PIi-fCQvX7S1I2xM3rJol0zQr6Uy5AXr20ftWYLN4Ls8-en59yYA6dcGI9oXswbuIZaITfY674PQC2m8u_HkGCNmRiYI Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:39:42.008882 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 1de97cce55e944dbb62a920ea11734f5 Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:42.010709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': 'efb6e5c0-4ef2-41af-8d0b-a410db398d96'} Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:39:42.011781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. Aug 24 10:39:42.013545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:42.015023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Returning 404 to user: Instance efb6e5c0-4ef2-41af-8d0b-a410db398d96 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:39:42.015023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6fbc1f27-9564-47fc-8a9a-05af39b82c03 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96" status: 404 len: 111 microversion: 2.1 time: 0.031045 Aug 24 10:39:42.015023 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1490/3110] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:41 2022] GET /compute/v2.1/servers/efb6e5c0-4ef2-41af-8d0b-a410db398d96 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:39:43.149736 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:43.150275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:43.156825 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:43.157314 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:43.157757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:43.158454 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:43.158913 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:43.159349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:43.173672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] HTTP exception thrown: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Request method failure captured: Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 HTTP/1.1 Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfei3ERPSR8lry9h62i-mTjfLEiY_G0pf4TOSE4I4STV_SQYl2_UIncspaIl58uBJJzOdoQ7pLY4XfcLnM6PIi-fCQvX7S1I2xM3rJol0zQr6Uy5AXr20ftWYLN4Ls8-en59yYA6dcGI9oXswbuIZaITfY674PQC2m8u_HkGCNmRiYI Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:39:43.175240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 1de97cce55e944dbb62a920ea11734f5 Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.177224 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '5692cc07-3f16-47a4-9993-503b451d9951'} Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:39:43.178309 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.180174 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.182730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Returning 404 to user: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:39:43.182730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-739ca3a7-5fd3-4595-8b17-b16ff26ed2f3 tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "DELETE /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 404 len: 111 microversion: 2.1 time: 0.030873 Aug 24 10:39:43.182730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1621/3111] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:39:43 2022] DELETE /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:39:43.185635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:43.186242 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] [instance: 5692cc07-3f16-47a4-9993-503b451d9951] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:43.192737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:43.193226 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:43.193666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:43.194356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:43.194814 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:43.195243 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:43.208215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] HTTP exception thrown: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Request method failure captured: Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 HTTP/1.1 Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfei3ERPSR8lry9h62i-mTjfLEiY_G0pf4TOSE4I4STV_SQYl2_UIncspaIl58uBJJzOdoQ7pLY4XfcLnM6PIi-fCQvX7S1I2xM3rJol0zQr6Uy5AXr20ftWYLN4Ls8-en59yYA6dcGI9oXswbuIZaITfY674PQC2m8u_HkGCNmRiYI Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:39:43.209736 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/e055ba4967db42108c069ebc3843dcf6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: e055ba4967db42108c069ebc3843dcf6 Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1270832268 Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 1de97cce55e944dbb62a920ea11734f5 Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeTestJSON-1270832268-project Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.211612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '5692cc07-3f16-47a4-9993-503b451d9951'} Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:39:43.212713 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. Aug 24 10:39:43.214300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:39:43.215797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] Returning 404 to user: Instance 5692cc07-3f16-47a4-9993-503b451d9951 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:39:43.215797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2193f125-a6ec-472d-bbe7-1bc39008742c tempest-AttachVolumeTestJSON-1270832268 tempest-AttachVolumeTestJSON-1270832268-project] 10.222.0.86 "GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951" status: 404 len: 111 microversion: 2.1 time: 0.029805 Aug 24 10:39:43.215797 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1491/3112] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:43 2022] GET /compute/v2.1/servers/5692cc07-3f16-47a4-9993-503b451d9951 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:39:51.429028 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f95b5049-db74-440c-a69e-d03fcd6893cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: .version_select at 0x7fa33d7c6f70>, body: {"keypair": {"name": "tempest-keypair-1026775799", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOdQXJWumqdKi3E8YBi1lg4Ky6ft5v0xiYp9RQ1znD6FcEf+9VNJ9Z78yYyb6g+D6r/mfiS5BHQ99EyCbBGna0xEOYVUyjL4UC7uNxhV+okis+4RzFtRje932Tuamgk1+w=="}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:39:51.436508 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=30,nova_api:UPDATE=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:51.447723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-f95b5049-db74-440c-a69e-d03fcd6893cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for project 9569b373f21841e8883b0b269073121a. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:39:51.451967 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-f95b5049-db74-440c-a69e-d03fcd6893cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for user 311b0a8b22174a278793537963b5d142 and project 9569b373f21841e8883b0b269073121a. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:39:51.476732 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f95b5049-db74-440c-a69e-d03fcd6893cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.164787 Aug 24 10:39:51.477793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1622/3113] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:39:51 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:52.829751 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1342708491", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "09fed119-b640-4458-a3fb-a9a45329f34d"}], "security_groups": [{"name": "tempest-securitygroup--763813045"}], "key_name": "tempest-keypair-1026775799", "user_data": "}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:39:52.838804 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=28,nova_api:DELETE=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:52.936920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] validate_networks() for [('09fed119-b640-4458-a3fb-a9a45329f34d', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:39:53.199752 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=71,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:53.310681 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:39:53.311110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:39:53.311585 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:39:53.311997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:39:53.312447 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:39:53.321392 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:53.321878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:53.322311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:53.323039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:53.323498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:53.323926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:53.341740 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for project 9569b373f21841e8883b0b269073121a. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:39:53.345846 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for user 311b0a8b22174a278793537963b5d142 and project 9569b373f21841e8883b0b269073121a. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:39:53.357401 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:39:53.359872 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:39:53.410423 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5d2d070a-f9c4-4377-8c29-cac9718fe11f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.584953 Aug 24 10:39:53.417870 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1492/3114] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:39:52 2022] POST /compute/v2.1/servers => generated 401 bytes in 586 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:39:53.420830 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=67,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:39:53.426922 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-56ca3601-0dec-42ad-a2a5-bcec494dac73 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:53.427170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-56ca3601-0dec-42ad-a2a5-bcec494dac73 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:53.491954 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-56ca3601-0dec-42ad-a2a5-bcec494dac73 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:53.504635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-56ca3601-0dec-42ad-a2a5-bcec494dac73 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1242 microversion: 2.1 time: 0.083410 Aug 24 10:39:53.504635 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1623/3115] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:53 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1242 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:39:54.524885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:54.525501 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:54.532506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:54.533190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:54.533628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:54.534317 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:54.534773 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:54.535195 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:54.598703 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:54.607197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1d19f470-9fdb-4ac8-85c6-30b7f3a213cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1294 microversion: 2.1 time: 0.087230 Aug 24 10:39:54.607762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1493/3116] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:54 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1294 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:39:55.628492 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:55.629116 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:55.637741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:55.638237 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:55.638708 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:55.639424 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:55.639870 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:55.640342 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:56.081000 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:56.089245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7faade3e-3240-488d-bbc9-ac28067e91ad tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1367 microversion: 2.1 time: 0.464617 Aug 24 10:39:56.089728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1624/3117] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:55 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1367 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:39:56.366819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-5a5d88f4-449f-4bb6-b71d-75a6f9978079 req-e3c1179f-6448-4870-a24e-d8ce9b31b7e8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee", "tag": "d7dd88f2-155f-4981-89e4-4678c3dbc793"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:39:56.378190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5a5d88f4-449f-4bb6-b71d-75a6f9978079 req-e3c1179f-6448-4870-a24e-d8ce9b31b7e8 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:56.378679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5a5d88f4-449f-4bb6-b71d-75a6f9978079 req-e3c1179f-6448-4870-a24e-d8ce9b31b7e8 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:56.379116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-5a5d88f4-449f-4bb6-b71d-75a6f9978079 req-e3c1179f-6448-4870-a24e-d8ce9b31b7e8 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:56.391337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-5a5d88f4-449f-4bb6-b71d-75a6f9978079 req-e3c1179f-6448-4870-a24e-d8ce9b31b7e8 service nova] Creating event network-changed:d7dd88f2-155f-4981-89e4-4678c3dbc793 for instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee on hv-cinder-85427 Aug 24 10:39:56.396199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-5a5d88f4-449f-4bb6-b71d-75a6f9978079 req-e3c1179f-6448-4870-a24e-d8ce9b31b7e8 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033551 Aug 24 10:39:56.396945 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1494/3118] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:39:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:39:57.110874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:57.111408 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:57.118018 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:57.118505 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:57.118949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:57.119641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:57.120439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:57.120642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:57.248410 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:57.255947 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d1e875d8-54bc-450d-b30d-64cd6564a6c3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1367 microversion: 2.1 time: 0.149064 Aug 24 10:39:57.256530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1625/3119] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:57 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1367 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:39:58.277613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:58.278608 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:58.284885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:58.285362 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:58.285790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:58.286473 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:58.286918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:58.287334 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:58.422356 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:58.430860 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7b726954-0680-466b-a32a-8796a8993766 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1367 microversion: 2.1 time: 0.157986 Aug 24 10:39:58.431420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1495/3120] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:58 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1367 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:39:59.450743 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:39:59.451361 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:39:59.457800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:59.458285 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:59.458730 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:59.459421 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:39:59.459882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:39:59.460679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:39:59.589087 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:39:59.596783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-6b26d6eb-eaab-4c99-8613-3310cdbd5144 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1367 microversion: 2.1 time: 0.149921 Aug 24 10:39:59.597349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1626/3121] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:39:59 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:00.618045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:00.618658 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:00.625653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:00.626155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:00.626574 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:00.627265 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:00.627715 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:00.628168 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:00.759835 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:00.767961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3bfe07c4-ae62-4a1f-8513-8edee833d3e4 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1367 microversion: 2.1 time: 0.153799 Aug 24 10:40:00.768577 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1496/3122] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:00 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:01.789213 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:01.789829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:01.796353 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:01.796835 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:01.797272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:01.797959 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:01.798412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:01.798841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:01.931449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:01.939719 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ba7dabe6-0424-4994-b478-70375a69c8c0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1367 microversion: 2.1 time: 0.154690 Aug 24 10:40:01.940321 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1627/3123] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:01 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:02.751631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-717a3f40-5609-4c25-ab4c-0aca47d9c40a req-812af1d6-b27a-4ced-81ec-609863b5c747 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee", "name": "network-vif-plugged", "status": "completed", "tag": "d7dd88f2-155f-4981-89e4-4678c3dbc793"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:40:02.762609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-717a3f40-5609-4c25-ab4c-0aca47d9c40a req-812af1d6-b27a-4ced-81ec-609863b5c747 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:02.762793 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-717a3f40-5609-4c25-ab4c-0aca47d9c40a req-812af1d6-b27a-4ced-81ec-609863b5c747 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:02.763218 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-717a3f40-5609-4c25-ab4c-0aca47d9c40a req-812af1d6-b27a-4ced-81ec-609863b5c747 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:02.774551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-717a3f40-5609-4c25-ab4c-0aca47d9c40a req-812af1d6-b27a-4ced-81ec-609863b5c747 service nova] Creating event network-vif-plugged:d7dd88f2-155f-4981-89e4-4678c3dbc793 for instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee on hv-cinder-85427 Aug 24 10:40:02.780404 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-717a3f40-5609-4c25-ab4c-0aca47d9c40a req-812af1d6-b27a-4ced-81ec-609863b5c747 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093071 Aug 24 10:40:02.781030 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1497/3124] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:40:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:40:02.960752 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:02.961369 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:02.968156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:02.968648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:02.969091 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:02.969786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:02.970244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:02.970673 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:03.126508 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:03.134227 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9f091492-fd30-4669-b84c-6affb892a5eb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1367 microversion: 2.1 time: 0.177434 Aug 24 10:40:03.134943 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1628/3125] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:02 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1367 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:04.155342 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:04.156038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:04.157999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:04.158666 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=6 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:04.166290 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:04.167538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:04.167986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:04.168701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:04.169156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:04.169656 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:04.305270 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:04.313530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b0149dbb-2a4b-49d5-aef5-d060229c0dd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1367 microversion: 2.1 time: 0.162309 Aug 24 10:40:04.314273 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1498/3126] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:04 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1367 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:05.334596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:05.335195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:05.337090 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=6 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:05.342619 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:05.343117 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:05.343548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:05.344259 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:05.344706 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:05.345125 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:05.526292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:05.534104 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-95f03249-0cf5-41e1-bf66-1659eb373951 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1557 microversion: 2.1 time: 0.203462 Aug 24 10:40:05.534656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1629/3127] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:05 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1557 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:05.554865 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5d3baaf6-15ae-4137-a006-b442fe0d8e58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:05.555353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5d3baaf6-15ae-4137-a006-b442fe0d8e58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:05.561986 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d3baaf6-15ae-4137-a006-b442fe0d8e58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:05.562466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d3baaf6-15ae-4137-a006-b442fe0d8e58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:05.562902 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d3baaf6-15ae-4137-a006-b442fe0d8e58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:05.563586 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d3baaf6-15ae-4137-a006-b442fe0d8e58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:05.564088 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d3baaf6-15ae-4137-a006-b442fe0d8e58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:05.564510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5d3baaf6-15ae-4137-a006-b442fe0d8e58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:05.639604 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5d3baaf6-15ae-4137-a006-b442fe0d8e58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.088522 Aug 24 10:40:05.640258 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1499/3128] 10.222.0.86 () {58 vars in 1275 bytes} [Wed Aug 24 10:40:05 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-interface => generated 284 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:40:06.802240 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-0027f038-b9a1-481b-881b-a237ae34e38c req-7646112f-c959-4346-8756-1f57075b3187 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee", "tag": "d7dd88f2-155f-4981-89e4-4678c3dbc793"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:40:06.812799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-0027f038-b9a1-481b-881b-a237ae34e38c req-7646112f-c959-4346-8756-1f57075b3187 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:06.813282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-0027f038-b9a1-481b-881b-a237ae34e38c req-7646112f-c959-4346-8756-1f57075b3187 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:06.813720 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-0027f038-b9a1-481b-881b-a237ae34e38c req-7646112f-c959-4346-8756-1f57075b3187 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:06.825751 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-0027f038-b9a1-481b-881b-a237ae34e38c req-7646112f-c959-4346-8756-1f57075b3187 service nova] Creating event network-changed:d7dd88f2-155f-4981-89e4-4678c3dbc793 for instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee on hv-cinder-85427 Aug 24 10:40:06.830800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-0027f038-b9a1-481b-881b-a237ae34e38c req-7646112f-c959-4346-8756-1f57075b3187 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032256 Aug 24 10:40:06.831554 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1630/3129] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:40:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:40:42.966484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=28,nova_api:INSERT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:42.967234 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=34 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:42.971533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b5d206c9-2b49-443b-aad8-f3ba1017410f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:42.972026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b5d206c9-2b49-443b-aad8-f3ba1017410f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:42.979378 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5d206c9-2b49-443b-aad8-f3ba1017410f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:42.979867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5d206c9-2b49-443b-aad8-f3ba1017410f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:42.980327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5d206c9-2b49-443b-aad8-f3ba1017410f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:42.981025 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5d206c9-2b49-443b-aad8-f3ba1017410f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:42.982035 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5d206c9-2b49-443b-aad8-f3ba1017410f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:42.982461 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b5d206c9-2b49-443b-aad8-f3ba1017410f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:43.006820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b5d206c9-2b49-443b-aad8-f3ba1017410f tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/ips" status: 200 len: 150 microversion: 2.1 time: 0.041854 Aug 24 10:40:43.007406 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1500/3130] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:40:42 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/ips => generated 150 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:40:44.382539 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=28,nova_api:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:44.383458 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=38 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:44.388241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-5bbd24ed-8f8a-4e2c-9abd-bb62c65dc318 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6582ff12-0dfb-4401-80f4-1afb6fb1ecec"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:40:44.390609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-5bbd24ed-8f8a-4e2c-9abd-bb62c65dc318 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:44.397953 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbd24ed-8f8a-4e2c-9abd-bb62c65dc318 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:44.398440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbd24ed-8f8a-4e2c-9abd-bb62c65dc318 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:44.398881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbd24ed-8f8a-4e2c-9abd-bb62c65dc318 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:44.399570 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbd24ed-8f8a-4e2c-9abd-bb62c65dc318 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:44.400027 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbd24ed-8f8a-4e2c-9abd-bb62c65dc318 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:44.400491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-5bbd24ed-8f8a-4e2c-9abd-bb62c65dc318 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:44.636916 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-5bbd24ed-8f8a-4e2c-9abd-bb62c65dc318 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.255792 Aug 24 10:40:44.637711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1631/3131] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:40:44 2022] POST /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-volume_attachments => generated 194 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:40:52.160467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-691ca408-3859-4794-b716-ae4f4a0e5424 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: .version_select at 0x7fa3401d8ca0>, body: {"keypair": {"name": "tempest-keypair-1152324369", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCF7XIav+0EvDwWqDsNZnAqAj2YLeBb3Ij3feXrsq3ztoWtIyOXXZaELoN6gcODEy4pSMmFEl0xZ3ACQEc7Vz1wGTkL+nJZ7oK4sKUGe5XCIFzQIfvQj8xP1LmL9QUv3Ww=="}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:40:52.176556 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-691ca408-3859-4794-b716-ae4f4a0e5424 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for project 9569b373f21841e8883b0b269073121a. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:40:52.180477 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-691ca408-3859-4794-b716-ae4f4a0e5424 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for user 311b0a8b22174a278793537963b5d142 and project 9569b373f21841e8883b0b269073121a. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:40:52.202376 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-691ca408-3859-4794-b716-ae4f4a0e5424 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.045949 Aug 24 10:40:52.202991 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1501/3132] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:40:52 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:40:53.420509 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-930863795", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "09fed119-b640-4458-a3fb-a9a45329f34d"}], "security_groups": [{"name": "tempest-securitygroup--755674033"}], "key_name": "tempest-keypair-1152324369", "user_data": "}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:40:53.522851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] validate_networks() for [('09fed119-b640-4458-a3fb-a9a45329f34d', None, None, None, None, None)] {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:40:53.940843 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:40:53.941294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:40:53.941759 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:40:53.942165 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:40:53.942573 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:40:53.951039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:53.951513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:53.951941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:53.952691 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:53.953142 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:53.953562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:53.970011 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for project 9569b373f21841e8883b0b269073121a. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:40:53.973889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for user 311b0a8b22174a278793537963b5d142 and project 9569b373f21841e8883b0b269073121a. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:40:53.985049 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:40:53.987386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:40:54.033555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a84a24d3-cda1-4a15-bee8-2a3b5a48a057 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.618912 Aug 24 10:40:54.034173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1632/3133] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:40:53 2022] POST /compute/v2.1/servers => generated 401 bytes in 620 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:40:54.042031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:40:54.046845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4bc0f8b4-24ff-4c23-9e8f-a8302b394c37 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:54.047458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4bc0f8b4-24ff-4c23-9e8f-a8302b394c37 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:54.169576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4bc0f8b4-24ff-4c23-9e8f-a8302b394c37 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:54.177414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4bc0f8b4-24ff-4c23-9e8f-a8302b394c37 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1241 microversion: 2.1 time: 0.136416 Aug 24 10:40:54.177962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1502/3134] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:54 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1241 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:40:55.198362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-634b50e9-1a96-4c8c-8d5f-5d8e50e2387c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:55.198976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-634b50e9-1a96-4c8c-8d5f-5d8e50e2387c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:55.252319 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-634b50e9-1a96-4c8c-8d5f-5d8e50e2387c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:55.259205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-634b50e9-1a96-4c8c-8d5f-5d8e50e2387c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1241 microversion: 2.1 time: 0.065235 Aug 24 10:40:55.259770 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1633/3135] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:55 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1241 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:56.278549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-54da5b7a-3b4c-4055-9382-9eaa014a7d05 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:56.279159 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-54da5b7a-3b4c-4055-9382-9eaa014a7d05 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:56.332252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-54da5b7a-3b4c-4055-9382-9eaa014a7d05 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:56.338929 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-54da5b7a-3b4c-4055-9382-9eaa014a7d05 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1241 microversion: 2.1 time: 0.064208 Aug 24 10:40:56.339515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1503/3136] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:56 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1241 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:57.359485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-13aa67f4-1e53-4fc1-99d3-a3f50728af5a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:57.360178 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-13aa67f4-1e53-4fc1-99d3-a3f50728af5a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:57.415032 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-13aa67f4-1e53-4fc1-99d3-a3f50728af5a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:57.423802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-13aa67f4-1e53-4fc1-99d3-a3f50728af5a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1241 microversion: 2.1 time: 0.068917 Aug 24 10:40:57.424437 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1634/3137] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:57 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1241 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:58.445013 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:58.445627 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:58.453303 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:58.453788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:58.454221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:58.454911 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:58.455417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:58.455842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:58.520802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:58.528417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8751c510-2109-4594-b9aa-9751641b7077 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1245 microversion: 2.1 time: 0.087264 Aug 24 10:40:58.528996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1504/3138] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:58 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1245 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:40:59.549434 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:40:59.550110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:40:59.557318 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:59.557805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:59.558275 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:59.558970 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:40:59.559429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:40:59.559879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:40:59.635901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:40:59.644186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7f5e8f32-34fb-4b46-8090-de5b0f33d4ed tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1299 microversion: 2.1 time: 0.098700 Aug 24 10:40:59.644753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1635/3139] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:40:59 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1299 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:00.472883 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-7680ca9a-3a6f-43ab-a829-c82f78f59b8f req-fc07f7c8-9af9-42a9-97f8-9b6405744f1e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8af08746-c7d3-4ce2-8a80-6c322bd3a181", "tag": "5c586991-188d-4f6b-9c11-9ddb2cbe2751"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:41:00.486398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-7680ca9a-3a6f-43ab-a829-c82f78f59b8f req-fc07f7c8-9af9-42a9-97f8-9b6405744f1e service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:00.486919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-7680ca9a-3a6f-43ab-a829-c82f78f59b8f req-fc07f7c8-9af9-42a9-97f8-9b6405744f1e service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:00.487354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-7680ca9a-3a6f-43ab-a829-c82f78f59b8f req-fc07f7c8-9af9-42a9-97f8-9b6405744f1e service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:00.503180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-7680ca9a-3a6f-43ab-a829-c82f78f59b8f req-fc07f7c8-9af9-42a9-97f8-9b6405744f1e service nova] Creating event network-changed:5c586991-188d-4f6b-9c11-9ddb2cbe2751 for instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 on hv-cinder-85427 Aug 24 10:41:00.508407 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-7680ca9a-3a6f-43ab-a829-c82f78f59b8f req-fc07f7c8-9af9-42a9-97f8-9b6405744f1e service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039583 Aug 24 10:41:00.509360 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1505/3140] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:41:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:41:00.665439 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:00.666098 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:00.673344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:00.673840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:00.674325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:00.675029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:00.675489 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:00.675924 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:00.903079 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:00.911097 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c80decba-6c83-4237-b6d8-3aab3a88382c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1366 microversion: 2.1 time: 0.249506 Aug 24 10:41:00.911679 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1636/3141] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:00 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1366 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:01.933216 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:01.933938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:01.941180 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:01.941600 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:01.942046 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:01.942762 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:01.943220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:01.943643 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:02.074970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:02.083352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-70c7299e-2f47-4dae-a8cc-34e3942293f8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1366 microversion: 2.1 time: 0.154450 Aug 24 10:41:02.083926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1506/3142] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:01 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:03.103920 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:03.104568 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:03.110963 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:03.111441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:03.111872 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:03.112576 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:03.113023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:03.113440 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:03.240592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:03.248140 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3382cae3-6c73-43b1-926b-f8a0366bfbcd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1366 microversion: 2.1 time: 0.147977 Aug 24 10:41:03.248702 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1637/3143] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:03 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1366 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:04.269864 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:04.270478 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:04.276987 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:04.277468 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:04.277904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:04.278588 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:04.279534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:04.279955 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:04.409564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:04.417120 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7e3da5f1-1b76-4479-a82d-fbf472492286 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1366 microversion: 2.1 time: 0.151165 Aug 24 10:41:04.417920 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1507/3144] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:04 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:05.438780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:05.439389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:05.441358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:41:05.447359 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:05.447987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:05.448456 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:05.449270 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:05.449737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:05.450168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:05.580906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:05.588480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-2e7f957e-a5e8-4fe4-83cd-381ccb5820be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1366 microversion: 2.1 time: 0.153713 Aug 24 10:41:05.589046 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1638/3145] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:05 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:06.610453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:06.611066 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:06.613472 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=8 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:41:06.619060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:06.619545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:06.619988 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:06.620701 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:06.621155 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:06.621575 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:06.750832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:06.758551 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-87400231-e34a-4408-a350-042fdb990778 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1366 microversion: 2.1 time: 0.152656 Aug 24 10:41:06.759123 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1508/3146] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:06 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:41:07.045925 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-414f21ce-ed16-43d9-84b6-101a2403a788 req-b8f17cc2-0886-499a-b585-97cf22a86bfb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8af08746-c7d3-4ce2-8a80-6c322bd3a181", "name": "network-vif-plugged", "status": "completed", "tag": "5c586991-188d-4f6b-9c11-9ddb2cbe2751"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:41:07.056069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-414f21ce-ed16-43d9-84b6-101a2403a788 req-b8f17cc2-0886-499a-b585-97cf22a86bfb service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:07.056650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-414f21ce-ed16-43d9-84b6-101a2403a788 req-b8f17cc2-0886-499a-b585-97cf22a86bfb service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:07.057183 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-414f21ce-ed16-43d9-84b6-101a2403a788 req-b8f17cc2-0886-499a-b585-97cf22a86bfb service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:07.070829 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-414f21ce-ed16-43d9-84b6-101a2403a788 req-b8f17cc2-0886-499a-b585-97cf22a86bfb service nova] Creating event network-vif-plugged:5c586991-188d-4f6b-9c11-9ddb2cbe2751 for instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 on hv-cinder-85427 Aug 24 10:41:07.075899 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-414f21ce-ed16-43d9-84b6-101a2403a788 req-b8f17cc2-0886-499a-b585-97cf22a86bfb service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034398 Aug 24 10:41:07.077039 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1639/3147] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:41:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:41:07.778710 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:07.779326 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:07.785915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:07.786396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:07.786830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:07.787518 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:07.787968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:07.788493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:07.921370 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:07.929467 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a66182ec-8a98-42a3-a252-24f3ac2c4497 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1366 microversion: 2.1 time: 0.154522 Aug 24 10:41:07.930062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1509/3148] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:07 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:08.951654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:08.952373 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:08.954297 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=10 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:41:08.960299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:08.960760 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:08.961197 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:08.961881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:08.962324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:08.962744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:09.091245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:09.100187 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-524c853d-1107-4510-97fc-a10c5cd3e146 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1557 microversion: 2.1 time: 0.152815 Aug 24 10:41:09.100753 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1640/3149] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:08 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1557 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:09.120237 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-734537e2-d92f-4b54-baec-1a194a878b5c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:09.120723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-734537e2-d92f-4b54-baec-1a194a878b5c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:09.127220 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-734537e2-d92f-4b54-baec-1a194a878b5c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:09.127708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-734537e2-d92f-4b54-baec-1a194a878b5c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:09.128204 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-734537e2-d92f-4b54-baec-1a194a878b5c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:09.128897 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-734537e2-d92f-4b54-baec-1a194a878b5c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:09.129353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-734537e2-d92f-4b54-baec-1a194a878b5c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:09.129779 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-734537e2-d92f-4b54-baec-1a194a878b5c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:09.208238 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-734537e2-d92f-4b54-baec-1a194a878b5c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.091765 Aug 24 10:41:09.209062 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1510/3150] 10.222.0.86 () {58 vars in 1275 bytes} [Wed Aug 24 10:41:09 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181/os-interface => generated 285 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:41:10.115955 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-fd4dbd17-15c5-45de-8680-f793b9798c2b req-5ea0ef68-f194-4a78-9937-b35d31691ef9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8af08746-c7d3-4ce2-8a80-6c322bd3a181", "tag": "5c586991-188d-4f6b-9c11-9ddb2cbe2751"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:41:10.126928 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-fd4dbd17-15c5-45de-8680-f793b9798c2b req-5ea0ef68-f194-4a78-9937-b35d31691ef9 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:10.127621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-fd4dbd17-15c5-45de-8680-f793b9798c2b req-5ea0ef68-f194-4a78-9937-b35d31691ef9 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:10.128149 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-fd4dbd17-15c5-45de-8680-f793b9798c2b req-5ea0ef68-f194-4a78-9937-b35d31691ef9 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:10.139939 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-fd4dbd17-15c5-45de-8680-f793b9798c2b req-5ea0ef68-f194-4a78-9937-b35d31691ef9 service nova] Creating event network-changed:5c586991-188d-4f6b-9c11-9ddb2cbe2751 for instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 on hv-cinder-85427 Aug 24 10:41:10.145047 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-fd4dbd17-15c5-45de-8680-f793b9798c2b req-5ea0ef68-f194-4a78-9937-b35d31691ef9 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032728 Aug 24 10:41:10.145650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1641/3151] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:41:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:41:46.243927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:41:46.244712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=36 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:41:46.248944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-42649ab8-0eb6-4e34-b2ed-914f0034a821 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:46.249924 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-42649ab8-0eb6-4e34-b2ed-914f0034a821 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:46.257031 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42649ab8-0eb6-4e34-b2ed-914f0034a821 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:46.257516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42649ab8-0eb6-4e34-b2ed-914f0034a821 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:46.257971 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42649ab8-0eb6-4e34-b2ed-914f0034a821 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:46.258649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42649ab8-0eb6-4e34-b2ed-914f0034a821 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:46.259104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42649ab8-0eb6-4e34-b2ed-914f0034a821 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:46.259530 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-42649ab8-0eb6-4e34-b2ed-914f0034a821 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:46.283318 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-42649ab8-0eb6-4e34-b2ed-914f0034a821 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181/ips" status: 200 len: 151 microversion: 2.1 time: 0.040695 Aug 24 10:41:46.283892 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1511/3152] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:41:46 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181/ips => generated 151 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:41:46.294074 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:41:46.295066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=47,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:41:46.300164 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6582ff12-0dfb-4401-80f4-1afb6fb1ecec"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:41:46.302535 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:46.310907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:46.311431 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:46.312386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:46.313069 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:46.313521 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:46.313948 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:46.393851 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Invalid volume: volume 6582ff12-0dfb-4401-80f4-1afb6fb1ecec is already attached to instances: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: POST /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181/os-volume_attachments HTTP/1.1 Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 74 Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:41:46.395592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {"volumeAttachment": {"volumeId": "6582ff12-0dfb-4401-80f4-1afb6fb1ecec"}} Aug 24 10:41:46.396853 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Invalid volume: volume 6582ff12-0dfb-4401-80f4-1afb6fb1ecec is already attached to instances: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'server_id': '8af08746-c7d3-4ce2-8a80-6c322bd3a181', 'body': {'volumeAttachment': {'volumeId': '6582ff12-0dfb-4401-80f4-1afb6fb1ecec'}}} Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5113, in attach_volume Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4953, in _check_volume_already_attached Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 6582ff12-0dfb-4401-80f4-1afb6fb1ecec is already attached to instances: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:46.397621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 6582ff12-0dfb-4401-80f4-1afb6fb1ecec is already attached to instances: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:41:46.398605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:41:46.399605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:41:46.399605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:41:46.399605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:41:46.399605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 6582ff12-0dfb-4401-80f4-1afb6fb1ecec is already attached to instances: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee Aug 24 10:41:46.399605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:46.399605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 400 to user: Invalid volume: volume 6582ff12-0dfb-4401-80f4-1afb6fb1ecec is already attached to instances: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:41:46.399605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-1eed5a29-3ed0-4068-8997-ef0ed761f14a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.104984 Aug 24 10:41:46.399605 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1642/3153] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:41:46 2022] POST /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181/os-volume_attachments => generated 174 bytes in 106 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:41:46.405708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:46.406206 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:46.412121 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:46.412605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:46.413041 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:46.413721 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:46.414179 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:46.414603 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:46.441838 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:41:46.518099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6ddff7b9-7731-47f7-8771-961795e41dde tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 204 len: 0 microversion: 2.1 time: 0.115854 Aug 24 10:41:46.518904 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1512/3154] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:41:46 2022] DELETE /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:41:46.529044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:46.529665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:46.535617 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:46.536134 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:46.536582 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:46.537273 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:46.537762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:46.538190 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:46.674632 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:46.682633 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-0da80171-6144-4d63-a37f-6a728d4da856 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1682 microversion: 2.1 time: 0.159022 Aug 24 10:41:46.683186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1643/3155] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:46 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1682 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:47.705192 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:47.705950 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:47.712312 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:47.712798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:47.713233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:47.713921 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:47.714554 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:47.714783 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:47.848750 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:47.856547 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-2bccfc55-2ec2-43c3-a043-95556d67635b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1682 microversion: 2.1 time: 0.156395 Aug 24 10:41:47.857107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1513/3156] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:47 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1682 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:48.861300 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-eb084b7a-c65e-4379-937a-7ccd3cccc423 req-257d0a89-b897-40bc-bf18-5d3e99bf2d8e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8af08746-c7d3-4ce2-8a80-6c322bd3a181", "name": "network-vif-deleted", "tag": "5c586991-188d-4f6b-9c11-9ddb2cbe2751"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:41:48.873908 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-eb084b7a-c65e-4379-937a-7ccd3cccc423 req-257d0a89-b897-40bc-bf18-5d3e99bf2d8e service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:48.874642 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-eb084b7a-c65e-4379-937a-7ccd3cccc423 req-257d0a89-b897-40bc-bf18-5d3e99bf2d8e service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:48.875326 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-eb084b7a-c65e-4379-937a-7ccd3cccc423 req-257d0a89-b897-40bc-bf18-5d3e99bf2d8e service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:48.878278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:48.879104 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:48.885878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:48.886420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:48.887349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:48.887590 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:48.888072 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:48.888632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:48.889263 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-eb084b7a-c65e-4379-937a-7ccd3cccc423 req-257d0a89-b897-40bc-bf18-5d3e99bf2d8e service nova] Creating event network-vif-deleted:5c586991-188d-4f6b-9c11-9ddb2cbe2751 for instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 on hv-cinder-85427 Aug 24 10:41:48.897205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-eb084b7a-c65e-4379-937a-7ccd3cccc423 req-257d0a89-b897-40bc-bf18-5d3e99bf2d8e service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039333 Aug 24 10:41:48.899414 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1644/3157] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:41:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:41:48.966463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:48.978583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1f56f3f8-5dab-4204-b7ee-cfba2ab7de27 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 200 len: 1326 microversion: 2.1 time: 0.104307 Aug 24 10:41:48.979873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1514/3158] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:48 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 1326 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:41:50.000649 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:50.001343 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:50.008304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:50.008827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:50.009272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:50.009974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:50.010432 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:50.010861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:50.024119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 HTTP/1.1 Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:41:50.025741 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:41:50.026849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '8af08746-c7d3-4ce2-8a80-6c322bd3a181'} Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:41:50.027468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:41:50.028612 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:50.029726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:41:50.029726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-4631b65b-1132-4125-a238-75d3129a1d6d tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 404 len: 111 microversion: 2.1 time: 0.030932 Aug 24 10:41:50.029726 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1645/3159] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:49 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:41:50.036832 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-64361885-1cda-4f1d-a2ed-07070b2c59c8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '.version_select at 0x7fa33da85e50>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:50.056227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-64361885-1cda-4f1d-a2ed-07070b2c59c8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1152324369" status: 202 len: 0 microversion: 2.1 time: 0.023057 Aug 24 10:41:50.056830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1515/3160] 10.222.0.86 () {58 vars in 1221 bytes} [Wed Aug 24 10:41:50 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1152324369 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:41:50.717641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:50.718136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:50.723979 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:50.724491 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:50.724931 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:50.725609 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:50.726057 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:50.726480 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:50.807857 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lazy-loading 'services' on Instance uuid 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:41:50.877667 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-35ada0cf-cc3e-416e-8b4a-b1b31ce8bc39 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-volume_attachments/6582ff12-0dfb-4401-80f4-1afb6fb1ecec" status: 202 len: 0 microversion: 2.1 time: 0.163791 Aug 24 10:41:50.878256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1646/3161] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:41:50 2022] DELETE /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-volume_attachments/6582ff12-0dfb-4401-80f4-1afb6fb1ecec => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:41:50.886502 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-300b159b-41e9-4c61-8549-dd085d1bcbd7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:50.888688 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-300b159b-41e9-4c61-8549-dd085d1bcbd7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:50.895126 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300b159b-41e9-4c61-8549-dd085d1bcbd7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:50.895612 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300b159b-41e9-4c61-8549-dd085d1bcbd7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:50.896077 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300b159b-41e9-4c61-8549-dd085d1bcbd7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:50.896763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300b159b-41e9-4c61-8549-dd085d1bcbd7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:50.897219 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300b159b-41e9-4c61-8549-dd085d1bcbd7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:50.897802 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-300b159b-41e9-4c61-8549-dd085d1bcbd7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:50.928725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-300b159b-41e9-4c61-8549-dd085d1bcbd7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045881 Aug 24 10:41:50.929205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1516/3162] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:41:50 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:41:51.941547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9521d587-057d-46a4-9e52-3c97d0328af8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:51.943874 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9521d587-057d-46a4-9e52-3c97d0328af8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:51.950133 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9521d587-057d-46a4-9e52-3c97d0328af8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:51.950616 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9521d587-057d-46a4-9e52-3c97d0328af8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:51.951059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9521d587-057d-46a4-9e52-3c97d0328af8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:51.952356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9521d587-057d-46a4-9e52-3c97d0328af8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:51.952809 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9521d587-057d-46a4-9e52-3c97d0328af8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:51.953256 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9521d587-057d-46a4-9e52-3c97d0328af8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:51.982064 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9521d587-057d-46a4-9e52-3c97d0328af8 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045243 Aug 24 10:41:51.982604 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1647/3163] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:41:51 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 24 10:41:52.042399 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:52.042905 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:52.049496 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:52.049980 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:52.050420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:52.051102 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:52.051555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:52.051982 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:52.076344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:41:52.150089 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-510993d7-8fa1-4d51-b75b-36fa0adc58be tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 204 len: 0 microversion: 2.1 time: 0.111276 Aug 24 10:41:52.150664 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1517/3164] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:41:52 2022] DELETE /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:41:52.159589 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:52.160244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:52.166182 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:52.166669 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:52.167115 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:52.167805 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:52.168290 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:52.168722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:52.298220 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:52.305904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d8729e04-3b62-4cbf-9bf8-bc1533346d1b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1682 microversion: 2.1 time: 0.150851 Aug 24 10:41:52.306465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1648/3165] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:52 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1682 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:53.329365 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:53.330058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:53.336364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:53.336839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:53.337266 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:53.337939 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:53.338383 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:53.338798 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:53.471380 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:53.479255 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1dc4e3bf-dee3-4705-8004-0ad7a6fd0212 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1682 microversion: 2.1 time: 0.154618 Aug 24 10:41:53.479853 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1518/3166] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:53 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1682 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:54.501331 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:54.501884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:54.508235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:54.508722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:54.509163 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:54.509854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:54.510313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:54.510747 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:54.626127 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:54.645915 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a07b0634-1745-4a05-87e4-d5ddfca3955a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 200 len: 1615 microversion: 2.1 time: 0.149155 Aug 24 10:41:54.646584 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1649/3167] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:54 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 1615 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:41:54.718507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-2fa27e84-1b0c-4588-b21e-af1c291cdf98 req-4fd0557c-d3b7-41a0-9a19-1b1cb4768dae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee", "name": "network-vif-deleted", "tag": "d7dd88f2-155f-4981-89e4-4678c3dbc793"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:41:54.728842 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-2fa27e84-1b0c-4588-b21e-af1c291cdf98 req-4fd0557c-d3b7-41a0-9a19-1b1cb4768dae service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:54.729377 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-2fa27e84-1b0c-4588-b21e-af1c291cdf98 req-4fd0557c-d3b7-41a0-9a19-1b1cb4768dae service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:54.729807 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-2fa27e84-1b0c-4588-b21e-af1c291cdf98 req-4fd0557c-d3b7-41a0-9a19-1b1cb4768dae service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:54.743576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-2fa27e84-1b0c-4588-b21e-af1c291cdf98 req-4fd0557c-d3b7-41a0-9a19-1b1cb4768dae service nova] Creating event network-vif-deleted:d7dd88f2-155f-4981-89e4-4678c3dbc793 for instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee on hv-cinder-85427 Aug 24 10:41:54.750694 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-2fa27e84-1b0c-4588-b21e-af1c291cdf98 req-4fd0557c-d3b7-41a0-9a19-1b1cb4768dae service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036202 Aug 24 10:41:54.752623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1519/3168] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:41:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:41:55.667978 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:55.668699 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:55.676368 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:55.676860 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:55.677304 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:55.678009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:55.678471 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:55.678901 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:55.694247 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee HTTP/1.1 Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:41:55.697272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:41:55.698485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee'} Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:41:55.699754 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:41:55.701650 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:41:55.703330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:41:55.703330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c2d4a2fe-a70e-4c9d-8555-bf3c1bc80136 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 404 len: 111 microversion: 2.1 time: 0.035728 Aug 24 10:41:55.703330 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1650/3169] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:55 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:41:55.708190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-cfd4144b-1f71-49d2-a6b5-54b56831a7e1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '.version_select at 0x7fa342252af0>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:55.729001 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-cfd4144b-1f71-49d2-a6b5-54b56831a7e1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1026775799" status: 202 len: 0 microversion: 2.1 time: 0.024870 Aug 24 10:41:55.732776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1520/3170] 10.222.0.86 () {58 vars in 1221 bytes} [Wed Aug 24 10:41:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1026775799 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:41:56.355214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9f4b4d34-db34-4bff-8552-7d8334f84cb2 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: .version_select at 0x7fa33d7f38b0>, body: {"keypair": {"name": "tempest-keypair-1588392053", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJbiyudm6wTKllAc+PFYqIObd8bQBVLv+UhVolBW9MfueySRk8QwyKb9dyJUiKoNaMl856KWecT9IAqt00T9SRjxlxg+ai6tqn0odDyc9yv4aWmNkpB3XEmtAeq9UFB4uQ=="}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:41:56.370448 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-9f4b4d34-db34-4bff-8552-7d8334f84cb2 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for project 9569b373f21841e8883b0b269073121a. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:41:56.374416 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-9f4b4d34-db34-4bff-8552-7d8334f84cb2 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for user 311b0a8b22174a278793537963b5d142 and project 9569b373f21841e8883b0b269073121a. Resources: {'key_pairs'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:41:56.385841 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR oslo.messaging._drivers.impl_rabbit [None req-9f4b4d34-db34-4bff-8552-7d8334f84cb2 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [40a8ab96-31a4-4f8c-8622-e745554b108b] AMQP server on 10.222.0.86:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 24 10:41:57.393695 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [None req-9f4b4d34-db34-4bff-8552-7d8334f84cb2 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [40a8ab96-31a4-4f8c-8622-e745554b108b] Reconnected to AMQP server on 10.222.0.86:5672 via [amqp] client with port 36542. Aug 24 10:41:57.408402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9f4b4d34-db34-4bff-8552-7d8334f84cb2 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.057003 Aug 24 10:41:57.409012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1651/3171] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:41:56 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1058 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:41:58.612067 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1227521442", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "09fed119-b640-4458-a3fb-a9a45329f34d"}], "security_groups": [{"name": "tempest-securitygroup--1828618764"}], "key_name": "tempest-keypair-1588392053", "user_data": "}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:41:58.713145 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.network.neutron [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] validate_networks() for [('09fed119-b640-4458-a3fb-a9a45329f34d', None, None, None, None, None)] {{(pid=120654) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:41:59.078015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Flavor limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:41:59.078430 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Image limits 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:41:59.078895 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Flavor pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:41:59.079353 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Image pref 0:0:0 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:41:59.079757 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.virt.hardware [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120654) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:41:59.088623 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:59.089085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:59.089519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:59.090227 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:41:59.090675 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:41:59.091096 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:41:59.111776 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for project 9569b373f21841e8883b0b269073121a. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:41:59.116153 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for user 311b0a8b22174a278793537963b5d142 and project 9569b373f21841e8883b0b269073121a. Resources: {'cores', 'ram', 'instances'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:41:59.128248 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Going to run 1 instances... {{(pid=120654) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:41:59.130933 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120654) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:41:59.180332 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3dc80bb7-3d42-46d6-80ce-9fb8c9e1a6d7 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.573792 Aug 24 10:41:59.180977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1521/3172] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:41:58 2022] POST /compute/v2.1/servers => generated 402 bytes in 575 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:41:59.192761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d7af70b0-cd5c-43e5-9dfa-3d0105ac3967 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:41:59.193941 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d7af70b0-cd5c-43e5-9dfa-3d0105ac3967 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:41:59.254360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d7af70b0-cd5c-43e5-9dfa-3d0105ac3967 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:41:59.261281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d7af70b0-cd5c-43e5-9dfa-3d0105ac3967 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1242 microversion: 2.1 time: 0.072918 Aug 24 10:41:59.261837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1652/3173] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:41:59 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1242 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:42:00.286396 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:00.287655 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:00.297275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:00.297970 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:00.298363 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:00.299241 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:00.299616 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:00.300125 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:00.368967 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:42:00.376640 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4b553ed3-ae7b-4e7a-a969-022677b83a6a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1300 microversion: 2.1 time: 0.098545 Aug 24 10:42:00.377197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1522/3174] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:42:00 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1300 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:42:01.397665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:01.398214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:01.404788 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:01.405578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:01.406021 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:01.406716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:01.407166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:01.407586 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:01.624585 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:42:01.632316 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-d5b2f5ed-cc0b-465c-845a-e065b50879b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1368 microversion: 2.1 time: 0.238564 Aug 24 10:42:01.632893 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1653/3175] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:42:01 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1368 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:42:01.859674 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-cb17c67c-08ef-47f4-9aa4-a0b6821434a6 req-10d7825b-aecf-49fd-a89b-9009e19fa448 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "10a8d89c-8dad-488f-a9e3-4e8455592b4c", "tag": "644b75be-74b4-4fb6-bc02-4533e7ec70df"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:42:01.869725 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-cb17c67c-08ef-47f4-9aa4-a0b6821434a6 req-10d7825b-aecf-49fd-a89b-9009e19fa448 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:01.870199 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-cb17c67c-08ef-47f4-9aa4-a0b6821434a6 req-10d7825b-aecf-49fd-a89b-9009e19fa448 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:01.870626 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-cb17c67c-08ef-47f4-9aa4-a0b6821434a6 req-10d7825b-aecf-49fd-a89b-9009e19fa448 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:01.881737 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-cb17c67c-08ef-47f4-9aa4-a0b6821434a6 req-10d7825b-aecf-49fd-a89b-9009e19fa448 service nova] Creating event network-changed:644b75be-74b4-4fb6-bc02-4533e7ec70df for instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c on hv-cinder-85427 Aug 24 10:42:01.886564 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-cb17c67c-08ef-47f4-9aa4-a0b6821434a6 req-10d7825b-aecf-49fd-a89b-9009e19fa448 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031153 Aug 24 10:42:01.887165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1523/3176] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:42:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:42:02.653324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:02.653846 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:02.660277 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:02.660761 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:02.661198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:02.661879 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:02.662335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:02.662767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:02.791906 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:42:02.809095 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3c6b4dc5-2cf1-49e3-be9b-c57868eca155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1368 microversion: 2.1 time: 0.159622 Aug 24 10:42:02.809692 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1654/3177] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:42:02 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1368 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:42:03.831996 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:03.832723 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:03.839166 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:03.839649 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:03.840114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:03.840801 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:03.841252 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:03.841679 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:03.969156 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:42:03.976927 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6ad9c8a6-564c-4983-8cad-d247ecfedfbd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1368 microversion: 2.1 time: 0.150572 Aug 24 10:42:03.977498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1524/3178] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:42:03 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1368 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:42:04.997783 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:04.998357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:05.004797 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:05.005268 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:05.005711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:05.006404 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:05.006863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:05.007295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:05.140490 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:42:05.151031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c89e6c76-62e2-4fc5-bfba-bdbd96aae873 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1368 microversion: 2.1 time: 0.157153 Aug 24 10:42:05.151694 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1655/3179] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:42:04 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1368 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:42:06.172696 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:06.173311 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:06.180018 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:06.180540 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:06.180997 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:06.181685 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:06.182140 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:06.182571 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:06.313717 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:42:06.321780 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-21444953-34c6-4b39-9d36-5362bd92a3cd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1368 microversion: 2.1 time: 0.153786 Aug 24 10:42:06.322350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1525/3180] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:42:06 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:42:07.341961 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:07.342574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:07.348840 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:07.349329 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:07.349767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:07.350461 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:07.350918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:07.351347 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:07.408082 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-f64c1da3-a6b6-442f-bf74-87c127849597 req-c17bf83d-94ab-4e3a-a4a9-03be3d43805d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "10a8d89c-8dad-488f-a9e3-4e8455592b4c", "name": "network-vif-plugged", "status": "completed", "tag": "644b75be-74b4-4fb6-bc02-4533e7ec70df"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:42:07.418034 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f64c1da3-a6b6-442f-bf74-87c127849597 req-c17bf83d-94ab-4e3a-a4a9-03be3d43805d service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:07.419770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f64c1da3-a6b6-442f-bf74-87c127849597 req-c17bf83d-94ab-4e3a-a4a9-03be3d43805d service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:07.419770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-f64c1da3-a6b6-442f-bf74-87c127849597 req-c17bf83d-94ab-4e3a-a4a9-03be3d43805d service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:07.430867 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-f64c1da3-a6b6-442f-bf74-87c127849597 req-c17bf83d-94ab-4e3a-a4a9-03be3d43805d service nova] Creating event network-vif-plugged:644b75be-74b4-4fb6-bc02-4533e7ec70df for instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c on hv-cinder-85427 Aug 24 10:42:07.441084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-f64c1da3-a6b6-442f-bf74-87c127849597 req-c17bf83d-94ab-4e3a-a4a9-03be3d43805d service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031472 Aug 24 10:42:07.442068 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1526/3181] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:42:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:42:07.501798 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:42:07.509884 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-c740145a-05d9-41b3-8cc7-74dad968b1b0 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1368 microversion: 2.1 time: 0.171820 Aug 24 10:42:07.510447 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1656/3182] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:42:07 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1368 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:42:08.345343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:08.345957 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:08.352026 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:08.352543 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:08.352974 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:08.353652 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:08.354108 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:08.354533 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:08.369268 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] HTTP exception thrown: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Request method failure captured: Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: DELETE /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 HTTP/1.1 Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBfdZQXnG6AIb-u3RcQ-yc69u-eEldWbKWCPZZqv-mHhomvyxd5PLYukdAEAwdUc9jJ1RzIUDFCbcnMnB66M_oSu8G6Lefyyjl8IcxcKnFAn-rEO_qwm4ay-PFgPAx24qS8zgrjL-9XlNGS1kJV6YTlDCxziXPUzeoHs_B9C0DbAn1pE Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Openstack-Nova-Api-Version: 2.20 Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:42:08.370896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: df9b402589fa4020bc241a4b9c06dd91 Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.372432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1'} Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:42:08.373275 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.374639 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.376086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Returning 404 to user: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:42:08.376086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-38823d05-e199-4109-8ebb-499fd66abdab tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "DELETE /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 404 len: 111 microversion: 2.20 time: 0.031859 Aug 24 10:42:08.376086 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1527/3183] 10.222.0.86 () {60 vars in 1280 bytes} [Wed Aug 24 10:42:08 2022] DELETE /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:42:08.381155 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:08.381794 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] [instance: 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:08.388106 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:08.389119 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:08.389562 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:08.390280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:08.390739 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:08.391168 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:08.405211 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] HTTP exception thrown: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Request method failure captured: Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 HTTP/1.1 Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBfdZQXnG6AIb-u3RcQ-yc69u-eEldWbKWCPZZqv-mHhomvyxd5PLYukdAEAwdUc9jJ1RzIUDFCbcnMnB66M_oSu8G6Lefyyjl8IcxcKnFAn-rEO_qwm4ay-PFgPAx24qS8zgrjL-9XlNGS1kJV6YTlDCxziXPUzeoHs_B9C0DbAn1pE Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Openstack-Nova-Api-Version: 2.20 Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:42:08.406787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/7c1ad7a7198d4e78b9d03f00ce65e091"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 7c1ad7a7198d4e78b9d03f00ce65e091 Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-840217806 Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: df9b402589fa4020bc241a4b9c06dd91 Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeShelveTestJSON-840217806-project Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.408173 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1'} Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:42:08.408937 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. Aug 24 10:42:08.410162 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:42:08.411257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] Returning 404 to user: Instance 7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:42:08.411257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-e42445c7-47fe-473d-95d4-c5cb71516518 tempest-AttachVolumeShelveTestJSON-840217806 tempest-AttachVolumeShelveTestJSON-840217806-project] 10.222.0.86 "GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1" status: 404 len: 111 microversion: 2.20 time: 0.030918 Aug 24 10:42:08.411257 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1657/3184] 10.222.0.86 () {60 vars in 1277 bytes} [Wed Aug 24 10:42:08 2022] GET /compute/v2.1/servers/7daa0cb3-5e48-46bc-b98f-1f9d02ed89a1 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:42:08.531456 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:08.532114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:08.538413 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:08.538889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:08.539316 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:08.540070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:08.540515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:08.540932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:09.041977 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:42:09.050327 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b6ab8046-d285-4c76-af03-aeaed6011ffb tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1368 microversion: 2.1 time: 0.523227 Aug 24 10:42:09.050894 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1528/3185] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:42:08 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1368 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:42:10.071994 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:10.072550 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:10.074476 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=6 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:10.080882 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:10.081475 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:10.081938 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:10.082651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:10.083105 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:10.083523 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:10.226349 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:42:10.235029 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-80bdbadc-f3a7-4d20-9262-1b6e9b02506e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1559 microversion: 2.1 time: 0.166889 Aug 24 10:42:10.235603 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1658/3186] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:42:10 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1559 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:42:10.257774 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f25d7856-8963-442b-b97e-e64b8b6ee868 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:10.258770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f25d7856-8963-442b-b97e-e64b8b6ee868 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:10.261498 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:10.262143 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=6 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:10.270114 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f25d7856-8963-442b-b97e-e64b8b6ee868 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:10.270599 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f25d7856-8963-442b-b97e-e64b8b6ee868 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:10.271119 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f25d7856-8963-442b-b97e-e64b8b6ee868 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:10.271808 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f25d7856-8963-442b-b97e-e64b8b6ee868 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:10.272369 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f25d7856-8963-442b-b97e-e64b8b6ee868 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:10.272790 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-f25d7856-8963-442b-b97e-e64b8b6ee868 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:10.366422 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f25d7856-8963-442b-b97e-e64b8b6ee868 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.114843 Aug 24 10:42:10.367450 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1529/3187] 10.222.0.86 () {58 vars in 1275 bytes} [Wed Aug 24 10:42:10 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-interface => generated 285 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:42:11.394537 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-b7bce191-5886-4f61-84a1-1f6d7fc488a0 req-0a762ae4-b3ae-4345-ae88-26f10fcf39fe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "10a8d89c-8dad-488f-a9e3-4e8455592b4c", "tag": "644b75be-74b4-4fb6-bc02-4533e7ec70df"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:42:11.411411 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b7bce191-5886-4f61-84a1-1f6d7fc488a0 req-0a762ae4-b3ae-4345-ae88-26f10fcf39fe service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:11.412157 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b7bce191-5886-4f61-84a1-1f6d7fc488a0 req-0a762ae4-b3ae-4345-ae88-26f10fcf39fe service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:11.412762 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-b7bce191-5886-4f61-84a1-1f6d7fc488a0 req-0a762ae4-b3ae-4345-ae88-26f10fcf39fe service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:11.434323 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-b7bce191-5886-4f61-84a1-1f6d7fc488a0 req-0a762ae4-b3ae-4345-ae88-26f10fcf39fe service nova] Creating event network-changed:644b75be-74b4-4fb6-bc02-4533e7ec70df for instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c on hv-cinder-85427 Aug 24 10:42:11.439834 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-b7bce191-5886-4f61-84a1-1f6d7fc488a0 req-0a762ae4-b3ae-4345-ae88-26f10fcf39fe service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050062 Aug 24 10:42:11.441083 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1659/3188] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:42:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:42:55.093385 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=2,nova_api:DELETE=2,nova_api:INSERT=3 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:55.094305 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:55.099510 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-c68f99fc-83f7-44f9-bafd-465c4e5765f3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:42:55.100006 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-c68f99fc-83f7-44f9-bafd-465c4e5765f3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:55.116858 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c68f99fc-83f7-44f9-bafd-465c4e5765f3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:55.117357 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c68f99fc-83f7-44f9-bafd-465c4e5765f3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:55.117800 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c68f99fc-83f7-44f9-bafd-465c4e5765f3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:55.118549 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c68f99fc-83f7-44f9-bafd-465c4e5765f3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:55.118999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c68f99fc-83f7-44f9-bafd-465c4e5765f3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:55.119420 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-c68f99fc-83f7-44f9-bafd-465c4e5765f3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:55.148827 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-c68f99fc-83f7-44f9-bafd-465c4e5765f3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/ips" status: 200 len: 151 microversion: 2.1 time: 0.056710 Aug 24 10:42:55.150085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1530/3189] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:42:55 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/ips => generated 151 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:42:56.519666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:56.520436 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=79,nova_cell1:INSERT=1 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:42:56.524881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f3392370-479f-4ca2-9e8e-1cfaed212e78 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "450e089b-511f-4263-b0fa-beb0241d0350"}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:42:56.527294 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f3392370-479f-4ca2-9e8e-1cfaed212e78 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:42:56.535855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3392370-479f-4ca2-9e8e-1cfaed212e78 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:56.536420 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3392370-479f-4ca2-9e8e-1cfaed212e78 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:56.536866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3392370-479f-4ca2-9e8e-1cfaed212e78 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:56.537574 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3392370-479f-4ca2-9e8e-1cfaed212e78 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:42:56.538035 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3392370-479f-4ca2-9e8e-1cfaed212e78 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:42:56.538470 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f3392370-479f-4ca2-9e8e-1cfaed212e78 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:42:56.821907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f3392370-479f-4ca2-9e8e-1cfaed212e78 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.303715 Aug 24 10:42:56.822524 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1660/3190] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:42:56 2022] POST /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments => generated 194 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:43:06.470045 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "450e089b-511f-4263-b0fa-beb0241d0350"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:43:06.472432 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:06.474900 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=4 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:06.475523 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=2 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:06.480915 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:06.481393 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:06.481822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:06.482494 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:06.482941 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:06.483359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:06.563665 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Invalid volume: volume 450e089b-511f-4263-b0fa-beb0241d0350 is already attached to instances: 10a8d89c-8dad-488f-a9e3-4e8455592b4c Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: POST /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments HTTP/1.1 Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 74 Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:43:06.566021 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {"volumeAttachment": {"volumeId": "450e089b-511f-4263-b0fa-beb0241d0350"}} Aug 24 10:43:06.567373 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Invalid volume: volume 450e089b-511f-4263-b0fa-beb0241d0350 is already attached to instances: 10a8d89c-8dad-488f-a9e3-4e8455592b4c Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'server_id': '10a8d89c-8dad-488f-a9e3-4e8455592b4c', 'body': {'volumeAttachment': {'volumeId': '450e089b-511f-4263-b0fa-beb0241d0350'}}} Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5113, in attach_volume Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4953, in _check_volume_already_attached Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 450e089b-511f-4263-b0fa-beb0241d0350 is already attached to instances: 10a8d89c-8dad-488f-a9e3-4e8455592b4c Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:06.568078 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 450e089b-511f-4263-b0fa-beb0241d0350 is already attached to instances: 10a8d89c-8dad-488f-a9e3-4e8455592b4c Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:43:06.569103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:43:06.570133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:43:06.570133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:43:06.570133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:43:06.570133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 450e089b-511f-4263-b0fa-beb0241d0350 is already attached to instances: 10a8d89c-8dad-488f-a9e3-4e8455592b4c Aug 24 10:43:06.570133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:06.570133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 400 to user: Invalid volume: volume 450e089b-511f-4263-b0fa-beb0241d0350 is already attached to instances: 10a8d89c-8dad-488f-a9e3-4e8455592b4c {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:43:06.570133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-5f4a1a0a-ec22-4959-a69c-369a32634b18 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.102078 Aug 24 10:43:06.570133 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1531/3191] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:43:06 2022] POST /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments => generated 174 bytes in 103 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 24 10:43:06.630433 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:06.635222 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:06.635812 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:06.642186 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:06.642676 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:06.643154 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:06.643868 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:06.644358 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:06.644793 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:06.728415 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lazy-loading 'services' on Instance uuid 10a8d89c-8dad-488f-a9e3-4e8455592b4c {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:43:06.804236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-feb1e5ca-7c94-4e19-85e7-9c0bd2e1b40a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments/450e089b-511f-4263-b0fa-beb0241d0350" status: 202 len: 0 microversion: 2.1 time: 0.174961 Aug 24 10:43:06.804824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1661/3192] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:43:06 2022] DELETE /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments/450e089b-511f-4263-b0fa-beb0241d0350 => generated 0 bytes in 177 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:43:06.813215 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-8ed74d75-88f1-4e53-91c4-1a0ef04454ac tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:06.815538 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-8ed74d75-88f1-4e53-91c4-1a0ef04454ac tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:06.821836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed74d75-88f1-4e53-91c4-1a0ef04454ac tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:06.822328 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed74d75-88f1-4e53-91c4-1a0ef04454ac tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:06.822792 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed74d75-88f1-4e53-91c4-1a0ef04454ac tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:06.823567 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed74d75-88f1-4e53-91c4-1a0ef04454ac tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:06.824065 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed74d75-88f1-4e53-91c4-1a0ef04454ac tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:06.824503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-8ed74d75-88f1-4e53-91c4-1a0ef04454ac tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:06.856630 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-8ed74d75-88f1-4e53-91c4-1a0ef04454ac tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047416 Aug 24 10:43:06.857190 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1532/3193] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:43:06 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:43:07.870122 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-19ff59c6-13f6-4313-8430-3cae2c61d597 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:07.872449 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-19ff59c6-13f6-4313-8430-3cae2c61d597 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:07.879170 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19ff59c6-13f6-4313-8430-3cae2c61d597 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:07.879644 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19ff59c6-13f6-4313-8430-3cae2c61d597 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:07.880101 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19ff59c6-13f6-4313-8430-3cae2c61d597 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:07.880780 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19ff59c6-13f6-4313-8430-3cae2c61d597 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:07.881235 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19ff59c6-13f6-4313-8430-3cae2c61d597 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:07.881654 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-19ff59c6-13f6-4313-8430-3cae2c61d597 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:07.911128 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-19ff59c6-13f6-4313-8430-3cae2c61d597 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045888 Aug 24 10:43:07.911668 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1662/3194] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:43:07 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:43:08.923926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bb970e56-cc13-4f34-b74f-c4ee027ed331 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:08.926708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bb970e56-cc13-4f34-b74f-c4ee027ed331 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:08.933014 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb970e56-cc13-4f34-b74f-c4ee027ed331 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:08.933488 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb970e56-cc13-4f34-b74f-c4ee027ed331 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:08.933917 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb970e56-cc13-4f34-b74f-c4ee027ed331 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:08.934591 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb970e56-cc13-4f34-b74f-c4ee027ed331 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:08.935039 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb970e56-cc13-4f34-b74f-c4ee027ed331 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:08.935458 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bb970e56-cc13-4f34-b74f-c4ee027ed331 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:08.963887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bb970e56-cc13-4f34-b74f-c4ee027ed331 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043946 Aug 24 10:43:08.964466 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1533/3195] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:43:08 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:43:09.978005 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-36a8702a-03e1-4c44-8f42-2ad7f977cbd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:09.980438 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-36a8702a-03e1-4c44-8f42-2ad7f977cbd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:09.986987 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36a8702a-03e1-4c44-8f42-2ad7f977cbd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:09.987510 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36a8702a-03e1-4c44-8f42-2ad7f977cbd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:09.988023 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36a8702a-03e1-4c44-8f42-2ad7f977cbd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:09.988799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36a8702a-03e1-4c44-8f42-2ad7f977cbd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:09.989344 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36a8702a-03e1-4c44-8f42-2ad7f977cbd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:09.989838 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-36a8702a-03e1-4c44-8f42-2ad7f977cbd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:10.022084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-36a8702a-03e1-4c44-8f42-2ad7f977cbd6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048275 Aug 24 10:43:10.022084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1663/3196] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:43:09 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:43:11.033887 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-326cca4d-17a2-4f70-a2d0-6a1da1ab806c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:11.035968 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-326cca4d-17a2-4f70-a2d0-6a1da1ab806c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:11.042460 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-326cca4d-17a2-4f70-a2d0-6a1da1ab806c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:11.042944 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-326cca4d-17a2-4f70-a2d0-6a1da1ab806c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:11.043632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-326cca4d-17a2-4f70-a2d0-6a1da1ab806c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:11.044459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-326cca4d-17a2-4f70-a2d0-6a1da1ab806c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:11.044918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-326cca4d-17a2-4f70-a2d0-6a1da1ab806c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:11.045350 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-326cca4d-17a2-4f70-a2d0-6a1da1ab806c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:11.075306 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-326cca4d-17a2-4f70-a2d0-6a1da1ab806c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045568 Aug 24 10:43:11.075857 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1534/3197] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:43:11 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 24 10:43:11.136777 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:11.137292 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:11.143593 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:11.144103 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:11.144547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:11.145236 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:11.145687 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:11.146121 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:11.173003 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Going to try to terminate instance {{(pid=120653) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:43:11.250975 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-7dd6fd1d-453f-45a5-b5eb-4d9b15ce6e7c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 204 len: 0 microversion: 2.1 time: 0.117894 Aug 24 10:43:11.251566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1664/3198] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:43:11 2022] DELETE /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:43:11.262300 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:11.262918 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:11.268749 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:11.269222 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:11.269651 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:11.270322 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:11.270769 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:11.271186 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:11.407932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:11.415830 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-bc048928-be1a-4e2d-939e-56deba933641 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1684 microversion: 2.1 time: 0.158626 Aug 24 10:43:11.416495 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1535/3199] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:11 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1684 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:12.439084 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:12.439775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:12.445971 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:12.446450 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:12.446892 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:12.447601 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:12.448092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:12.448493 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:12.577571 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:12.585711 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-84b81ebb-96c8-4a66-8fd6-da204b0a1e6b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1684 microversion: 2.1 time: 0.151046 Aug 24 10:43:12.586280 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1665/3200] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:12 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1684 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:13.607922 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:13.608605 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:13.616033 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:13.616662 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:13.617165 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:13.617932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:13.618534 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:13.619038 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:13.753555 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:13.761781 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-a64dee79-e488-4cf8-9f82-0b6a6c04f3dd tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1684 microversion: 2.1 time: 0.158219 Aug 24 10:43:13.762359 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1536/3201] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:13 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1684 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 24 10:43:14.284198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-36009fbc-7176-4919-a651-681e610e7b36 req-5689dc79-a53c-4f4b-82b2-a17416d990ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "10a8d89c-8dad-488f-a9e3-4e8455592b4c", "name": "network-vif-deleted", "tag": "644b75be-74b4-4fb6-bc02-4533e7ec70df"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:43:14.294750 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-36009fbc-7176-4919-a651-681e610e7b36 req-5689dc79-a53c-4f4b-82b2-a17416d990ba service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:14.295346 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-36009fbc-7176-4919-a651-681e610e7b36 req-5689dc79-a53c-4f4b-82b2-a17416d990ba service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:14.295849 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-36009fbc-7176-4919-a651-681e610e7b36 req-5689dc79-a53c-4f4b-82b2-a17416d990ba service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:14.309607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-36009fbc-7176-4919-a651-681e610e7b36 req-5689dc79-a53c-4f4b-82b2-a17416d990ba service nova] Creating event network-vif-deleted:644b75be-74b4-4fb6-bc02-4533e7ec70df for instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c on hv-cinder-85427 Aug 24 10:43:14.315014 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-36009fbc-7176-4919-a651-681e610e7b36 req-5689dc79-a53c-4f4b-82b2-a17416d990ba service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.166071 Aug 24 10:43:14.316245 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1666/3202] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:43:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:43:14.788646 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:14.790354 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:14.798622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:14.799148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:14.799609 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:14.801103 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:14.801613 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:14.801953 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:14.871402 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:14.879390 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d9f05bb8-a56f-4152-859b-9131348991da tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1327 microversion: 2.1 time: 0.100104 Aug 24 10:43:14.879962 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1537/3203] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:14 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1327 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:15.902053 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:15.902672 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:15.908816 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:15.909951 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:15.910396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:15.911093 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:15.911552 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:15.911986 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:15.975012 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:15.982700 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-dfadd96e-d448-431b-9180-dcb65297875b tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 200 len: 1327 microversion: 2.1 time: 0.085421 Aug 24 10:43:15.983264 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1667/3204] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:15 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 1327 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:17.003080 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:17.003821 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:17.010705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:17.011292 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:17.011735 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:17.012454 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:17.012919 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:17.013333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:17.026565 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c HTTP/1.1 Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:43:17.028116 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:43:17.029452 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '10a8d89c-8dad-488f-a9e3-4e8455592b4c'} Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:43:17.030198 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:43:17.031336 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:43:17.032499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:43:17.032499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b24810e8-96b5-47e9-b29b-59c44607ec2a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 404 len: 111 microversion: 2.1 time: 0.030412 Aug 24 10:43:17.032499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1538/3205] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:16 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:43:17.038771 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-b36f9eca-3cc4-486a-8df4-8a99c35e1c3c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '.version_select at 0x7fa33d82a3a0>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:17.057974 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-b36f9eca-3cc4-486a-8df4-8a99c35e1c3c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1588392053" status: 202 len: 0 microversion: 2.1 time: 0.022876 Aug 24 10:43:17.058507 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1668/3206] 10.222.0.86 () {58 vars in 1221 bytes} [Wed Aug 24 10:43:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1588392053 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:43:17.655439 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-3913bc01-7835-4fa6-bbcf-ed33f46cc91c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: .version_select at 0x7fa33da24d30>, body: {"keypair": {"name": "tempest-keypair-164013944", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP+yl30crodooAji/LuEA0rxyJda2EdovLQ7XLy/96GNI78UNORBtN19D1exkkXbBhZ9Gr3435bIWFCgJ50KZksjCyXfqGMDDUL+Lrc/xjIbik6xoYvCSAjcjqxLpFb4UQ=="}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:43:17.670873 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-3913bc01-7835-4fa6-bbcf-ed33f46cc91c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for project 9569b373f21841e8883b0b269073121a. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:43:17.674628 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.quota [None req-3913bc01-7835-4fa6-bbcf-ed33f46cc91c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for user 311b0a8b22174a278793537963b5d142 and project 9569b373f21841e8883b0b269073121a. Resources: {'key_pairs'} {{(pid=120654) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:43:17.696486 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-3913bc01-7835-4fa6-bbcf-ed33f46cc91c tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.044870 Aug 24 10:43:17.697059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1539/3207] 10.222.0.86 () {60 vars in 1159 bytes} [Wed Aug 24 10:43:17 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:43:19.077666 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2009110217", "imageRef": "e371b603-1455-49dc-bf2a-5acf95571218", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "09fed119-b640-4458-a3fb-a9a45329f34d"}], "security_groups": [{"name": "tempest-securitygroup--1381767308"}], "key_name": "tempest-keypair-164013944", "user_data": "}} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:43:19.181485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.network.neutron [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] validate_networks() for [('09fed119-b640-4458-a3fb-a9a45329f34d', None, None, None, None, None)] {{(pid=120653) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 24 10:43:19.568786 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Flavor limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 24 10:43:19.569478 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Image limits 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 24 10:43:19.570310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Flavor pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 24 10:43:19.570904 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Image pref 0:0:0 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 24 10:43:19.571592 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.virt.hardware [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120653) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 24 10:43:19.582324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:19.582799 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:19.583233 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:19.583944 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:19.584463 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:19.584887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:19.601333 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for project 9569b373f21841e8883b0b269073121a. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 24 10:43:19.605281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.quota [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Getting quotas for user 311b0a8b22174a278793537963b5d142 and project 9569b373f21841e8883b0b269073121a. Resources: {'cores', 'ram', 'instances'} {{(pid=120653) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 24 10:43:19.616548 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Going to run 1 instances... {{(pid=120653) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 24 10:43:19.618822 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='e371b603-1455-49dc-bf2a-5acf95571218',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120653) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 24 10:43:19.665957 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-be46ac67-dfa8-47bb-85ee-b69c2b14f4a3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.592820 Aug 24 10:43:19.666824 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1669/3208] 10.222.0.86 () {60 vars in 1147 bytes} [Wed Aug 24 10:43:19 2022] POST /compute/v2.1/servers => generated 402 bytes in 595 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 24 10:43:19.679110 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-f6dcf7de-71a9-4cc1-ac19-e8ba06361ff1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:19.679912 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-f6dcf7de-71a9-4cc1-ac19-e8ba06361ff1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:19.740726 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-f6dcf7de-71a9-4cc1-ac19-e8ba06361ff1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:19.747788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-f6dcf7de-71a9-4cc1-ac19-e8ba06361ff1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1241 microversion: 2.1 time: 0.073776 Aug 24 10:43:19.748566 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1540/3209] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:19 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1241 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:20.769889 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:20.771453 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:20.779143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:20.779641 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:20.780166 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:20.780866 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:20.781325 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:20.781757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:20.871990 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:20.880689 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3f106ac7-b696-4967-954c-1df95bc2ef96 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1301 microversion: 2.1 time: 0.115756 Aug 24 10:43:20.881252 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1670/3210] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:20 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1301 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:21.902503 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:21.903233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:21.911441 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:21.912029 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:21.912673 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:21.913445 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:21.914070 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:21.914506 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:22.056975 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:22.066022 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-6d6a6c5b-d2f2-4b49-950c-0d020d56fc43 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1367 microversion: 2.1 time: 0.167343 Aug 24 10:43:22.068094 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1541/3211] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:21 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1367 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:22.345402 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-a55ec06e-c0c5-432e-93f1-f1bb47d315e4 req-ea0fef4d-05f5-4445-8f1b-9b5862377b6a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f51be366-af4a-492f-80b3-3c34b0f285ac", "tag": "09fbe61e-3bab-42df-be0e-2c00c0b7934b"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:43:22.361413 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a55ec06e-c0c5-432e-93f1-f1bb47d315e4 req-ea0fef4d-05f5-4445-8f1b-9b5862377b6a service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:22.362356 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a55ec06e-c0c5-432e-93f1-f1bb47d315e4 req-ea0fef4d-05f5-4445-8f1b-9b5862377b6a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:22.362949 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-a55ec06e-c0c5-432e-93f1-f1bb47d315e4 req-ea0fef4d-05f5-4445-8f1b-9b5862377b6a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:22.377232 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-a55ec06e-c0c5-432e-93f1-f1bb47d315e4 req-ea0fef4d-05f5-4445-8f1b-9b5862377b6a service nova] Creating event network-changed:09fbe61e-3bab-42df-be0e-2c00c0b7934b for instance f51be366-af4a-492f-80b3-3c34b0f285ac on hv-cinder-85427 Aug 24 10:43:22.383393 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-a55ec06e-c0c5-432e-93f1-f1bb47d315e4 req-ea0fef4d-05f5-4445-8f1b-9b5862377b6a service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043436 Aug 24 10:43:22.384396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1671/3212] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:43:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:43:23.087633 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:23.088194 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:23.094709 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:23.095191 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:23.095629 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:23.097028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:23.097484 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:23.097910 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:23.224788 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:23.232824 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-867dad31-e5d2-42a4-94ca-5ce0e0ce5f21 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1367 microversion: 2.1 time: 0.149156 Aug 24 10:43:23.233387 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1542/3213] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:23 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1367 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:24.253386 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:24.254044 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:24.260622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:24.261100 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:24.261530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:24.262208 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:24.262655 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:24.263073 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:24.390787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:24.398324 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-952231b1-df19-4159-8a44-419fdd9ac099 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1367 microversion: 2.1 time: 0.149179 Aug 24 10:43:24.398881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1672/3214] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:24 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:25.419443 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:25.420057 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:25.426463 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:25.426935 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:25.427364 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:25.428060 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:25.428507 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:25.428926 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:25.558743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:25.566289 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-fc83d61d-e8b1-4b2a-baa3-25143b9fcb58 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1367 microversion: 2.1 time: 0.150938 Aug 24 10:43:25.566845 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1543/3215] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:25 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1367 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:26.586864 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:26.587488 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:26.593861 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:26.594340 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:26.594778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:26.595466 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:26.595926 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:26.596876 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:26.600895 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:26.728891 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:26.737429 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-ddd9f76c-f0bf-40b2-8586-6710291edc32 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1367 microversion: 2.1 time: 0.153873 Aug 24 10:43:26.737996 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1673/3216] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:26 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:27.758093 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:27.758708 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:27.765250 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:27.765742 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:27.766183 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:27.766885 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:27.767343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:27.767772 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:27.817293 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [req-29f33839-fcb8-4d22-a3e0-2b83e94da8a0 req-ab1302f7-d36c-42b7-b1b5-535fecef7d27 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f51be366-af4a-492f-80b3-3c34b0f285ac", "name": "network-vif-plugged", "status": "completed", "tag": "09fbe61e-3bab-42df-be0e-2c00c0b7934b"}]} {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:43:27.828311 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-29f33839-fcb8-4d22-a3e0-2b83e94da8a0 req-ab1302f7-d36c-42b7-b1b5-535fecef7d27 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:27.829017 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-29f33839-fcb8-4d22-a3e0-2b83e94da8a0 req-ab1302f7-d36c-42b7-b1b5-535fecef7d27 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:27.829653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [req-29f33839-fcb8-4d22-a3e0-2b83e94da8a0 req-ab1302f7-d36c-42b7-b1b5-535fecef7d27 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:27.843310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.compute.server_external_events [req-29f33839-fcb8-4d22-a3e0-2b83e94da8a0 req-ab1302f7-d36c-42b7-b1b5-535fecef7d27 service nova] Creating event network-vif-plugged:09fbe61e-3bab-42df-be0e-2c00c0b7934b for instance f51be366-af4a-492f-80b3-3c34b0f285ac on hv-cinder-85427 Aug 24 10:43:27.851827 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [req-29f33839-fcb8-4d22-a3e0-2b83e94da8a0 req-ab1302f7-d36c-42b7-b1b5-535fecef7d27 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038195 Aug 24 10:43:27.853145 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1674/3217] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:43:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:43:27.912516 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:27.920559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e0c71ac6-ecaa-4d4a-969c-9530f0a97233 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1367 microversion: 2.1 time: 0.166406 Aug 24 10:43:27.921307 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1544/3218] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:27 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1367 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:28.942790 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:28.943482 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:28.949578 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:28.950051 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:28.950484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:28.951158 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:28.951607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:28.952026 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:29.078305 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:29.086723 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9bf198c3-de3f-4806-9e20-b01cba4d13b6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1367 microversion: 2.1 time: 0.148543 Aug 24 10:43:29.087317 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1675/3219] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:28 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1367 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:30.107886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:30.108519 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:30.111098 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell0:SELECT=6 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:30.117058 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:30.117531 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:30.117963 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:30.118635 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:30.119079 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:30.119493 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:30.250999 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:43:30.258861 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-dfb97e3b-f730-4ec0-b4af-4d716a4a030e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1557 microversion: 2.1 time: 0.154837 Aug 24 10:43:30.259421 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1545/3220] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:43:30 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1557 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:43:30.278722 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8ce56f3a-fddd-4704-9608-3e431f527335 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:43:30.279243 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8ce56f3a-fddd-4704-9608-3e431f527335 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:43:30.281218 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=2 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:30.282033 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell0:SELECT=6 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:43:30.288624 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ce56f3a-fddd-4704-9608-3e431f527335 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:30.289055 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ce56f3a-fddd-4704-9608-3e431f527335 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:30.289497 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ce56f3a-fddd-4704-9608-3e431f527335 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:30.290194 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ce56f3a-fddd-4704-9608-3e431f527335 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:30.290656 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ce56f3a-fddd-4704-9608-3e431f527335 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:30.291092 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8ce56f3a-fddd-4704-9608-3e431f527335 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:30.368412 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8ce56f3a-fddd-4704-9608-3e431f527335 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.093433 Aug 24 10:43:30.369031 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1676/3221] 10.222.0.86 () {58 vars in 1275 bytes} [Wed Aug 24 10:43:30 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:43:31.508148 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-83a20795-5c89-47f6-9328-01250233c10f req-3ea85ceb-457d-4c53-a0a4-4f7a192e2b13 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f51be366-af4a-492f-80b3-3c34b0f285ac", "tag": "09fbe61e-3bab-42df-be0e-2c00c0b7934b"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:43:31.519052 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-83a20795-5c89-47f6-9328-01250233c10f req-3ea85ceb-457d-4c53-a0a4-4f7a192e2b13 service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:43:31.519527 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-83a20795-5c89-47f6-9328-01250233c10f req-3ea85ceb-457d-4c53-a0a4-4f7a192e2b13 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:43:31.519949 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-83a20795-5c89-47f6-9328-01250233c10f req-3ea85ceb-457d-4c53-a0a4-4f7a192e2b13 service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:43:31.531016 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-83a20795-5c89-47f6-9328-01250233c10f req-3ea85ceb-457d-4c53-a0a4-4f7a192e2b13 service nova] Creating event network-changed:09fbe61e-3bab-42df-be0e-2c00c0b7934b for instance f51be366-af4a-492f-80b3-3c34b0f285ac on hv-cinder-85427 Aug 24 10:43:31.536028 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-83a20795-5c89-47f6-9328-01250233c10f req-3ea85ceb-457d-4c53-a0a4-4f7a192e2b13 service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031643 Aug 24 10:43:31.536766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1546/3222] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:43:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:44:07.600881 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:07.601659 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG dbcounter [-] [120653] Writing DB stats nova_cell1:SELECT=17 {{(pid=120653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:07.606153 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-9cf6a18b-ae4b-42a4-86e6-3bdad25f8dee tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:07.606648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-9cf6a18b-ae4b-42a4-86e6-3bdad25f8dee tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:07.614360 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cf6a18b-ae4b-42a4-86e6-3bdad25f8dee tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:07.614863 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cf6a18b-ae4b-42a4-86e6-3bdad25f8dee tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:07.615295 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cf6a18b-ae4b-42a4-86e6-3bdad25f8dee tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:07.615982 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cf6a18b-ae4b-42a4-86e6-3bdad25f8dee tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:07.616468 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cf6a18b-ae4b-42a4-86e6-3bdad25f8dee tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:07.616894 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-9cf6a18b-ae4b-42a4-86e6-3bdad25f8dee tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:07.640765 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-9cf6a18b-ae4b-42a4-86e6-3bdad25f8dee tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/ips" status: 200 len: 149 microversion: 2.1 time: 0.041444 Aug 24 10:44:07.641338 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1677/3223] 10.222.0.86 () {58 vars in 1248 bytes} [Wed Aug 24 10:44:07 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/ips => generated 149 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:44:08.967337 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-994882eb-58ab-47ce-915b-9b67a2a0d4e3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f6016fa1-871d-4e33-b07c-173ecb3850b2"}} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:44:08.969784 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-994882eb-58ab-47ce-915b-9b67a2a0d4e3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:08.972705 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_api:SELECT=43,nova_api:INSERT=1 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:08.973333 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG dbcounter [-] [120654] Writing DB stats nova_cell1:SELECT=76 {{(pid=120654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:44:08.979278 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-994882eb-58ab-47ce-915b-9b67a2a0d4e3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:08.979753 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-994882eb-58ab-47ce-915b-9b67a2a0d4e3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:08.980217 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-994882eb-58ab-47ce-915b-9b67a2a0d4e3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:08.980896 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-994882eb-58ab-47ce-915b-9b67a2a0d4e3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:08.981344 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-994882eb-58ab-47ce-915b-9b67a2a0d4e3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:08.981761 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-994882eb-58ab-47ce-915b-9b67a2a0d4e3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:09.221127 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-994882eb-58ab-47ce-915b-9b67a2a0d4e3 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "POST /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.257958 Aug 24 10:44:09.221766 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1547/3224] 10.222.0.86 () {60 vars in 1323 bytes} [Wed Aug 24 10:44:08 2022] POST /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-volume_attachments => generated 194 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:44:15.783143 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:15.783684 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:15.790628 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:15.791110 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:15.791547 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:15.792282 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:15.792745 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:15.793425 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:15.875737 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.objects.instance [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lazy-loading 'services' on Instance uuid f51be366-af4a-492f-80b3-3c34b0f285ac {{(pid=120653) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 24 10:44:15.948009 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-69a1d3b5-0c5e-4ccf-a4f5-48f12002954a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-volume_attachments/f6016fa1-871d-4e33-b07c-173ecb3850b2" status: 202 len: 0 microversion: 2.1 time: 0.168748 Aug 24 10:44:15.948614 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1678/3225] 10.222.0.86 () {58 vars in 1416 bytes} [Wed Aug 24 10:44:15 2022] DELETE /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-volume_attachments/f6016fa1-871d-4e33-b07c-173ecb3850b2 => generated 0 bytes in 170 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:44:15.958417 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-e0f6bacc-44de-4253-be6c-5fc15cd6264e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:15.960822 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-e0f6bacc-44de-4253-be6c-5fc15cd6264e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:15.967085 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0f6bacc-44de-4253-be6c-5fc15cd6264e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:15.967576 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0f6bacc-44de-4253-be6c-5fc15cd6264e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:15.968015 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0f6bacc-44de-4253-be6c-5fc15cd6264e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:15.968743 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0f6bacc-44de-4253-be6c-5fc15cd6264e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:15.969513 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0f6bacc-44de-4253-be6c-5fc15cd6264e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:15.969979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-e0f6bacc-44de-4253-be6c-5fc15cd6264e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:16.001205 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-e0f6bacc-44de-4253-be6c-5fc15cd6264e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048019 Aug 24 10:44:16.001760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1548/3226] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:44:15 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 24 10:44:17.014266 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-91bb2ad2-c5a4-445d-9838-df08ce2d0118 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:17.016651 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-91bb2ad2-c5a4-445d-9838-df08ce2d0118 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:17.022929 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91bb2ad2-c5a4-445d-9838-df08ce2d0118 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:17.023357 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91bb2ad2-c5a4-445d-9838-df08ce2d0118 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:17.023800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91bb2ad2-c5a4-445d-9838-df08ce2d0118 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:17.024513 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91bb2ad2-c5a4-445d-9838-df08ce2d0118 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:17.024966 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91bb2ad2-c5a4-445d-9838-df08ce2d0118 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:17.025391 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-91bb2ad2-c5a4-445d-9838-df08ce2d0118 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:17.054746 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-91bb2ad2-c5a4-445d-9838-df08ce2d0118 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045208 Aug 24 10:44:17.055281 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1679/3227] 10.222.0.86 () {58 vars in 1302 bytes} [Wed Aug 24 10:44:17 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 24 10:44:17.114826 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:17.115352 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:17.121661 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:17.122141 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:17.122583 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:17.123260 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:17.123712 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:17.124160 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:17.148233 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Going to try to terminate instance {{(pid=120654) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 24 10:44:17.223876 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-7873520d-f733-418a-a11b-1ad7bd85d807 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 204 len: 0 microversion: 2.1 time: 0.113134 Aug 24 10:44:17.224515 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1549/3228] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:44:17 2022] DELETE /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 24 10:44:17.232890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:17.233506 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:17.239962 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:17.240479 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:17.240918 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:17.241606 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:17.242061 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:17.242485 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:17.370775 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:44:17.378362 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8f302b79-d1cf-4ce0-a778-b2e04a596649 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1681 microversion: 2.1 time: 0.149381 Aug 24 10:44:17.378921 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1680/3229] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:44:17 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1681 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:44:18.400632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:18.401331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:18.407689 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:18.408209 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:18.408653 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:18.409992 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:18.410451 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:18.410878 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:18.540960 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.policy [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120654) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:44:18.548819 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-4cfa982f-2de9-4e50-9228-e1cc6fff93b1 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1681 microversion: 2.1 time: 0.152504 Aug 24 10:44:18.549372 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1550/3230] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:44:18 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1681 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:44:19.569890 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:19.570515 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:19.577136 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:19.577620 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:19.578059 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:19.578744 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:19.579198 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:19.579622 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:19.648008 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.policy [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '311b0a8b22174a278793537963b5d142', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9569b373f21841e8883b0b269073121a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120653) authorize /opt/stack/nova/nova/policy.py:199}} Aug 24 10:44:19.655625 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-afe2a0fd-5404-4b58-814f-5213e4f7c2a6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 200 len: 1613 microversion: 2.1 time: 0.089652 Aug 24 10:44:19.656205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1681/3231] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:44:19 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 1613 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 24 10:44:19.844019 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [req-ade08e2a-330f-46b2-9811-86dcec0930f3 req-27ae5b0e-faea-4234-9193-fae142fb011a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f51be366-af4a-492f-80b3-3c34b0f285ac", "name": "network-vif-deleted", "tag": "09fbe61e-3bab-42df-be0e-2c00c0b7934b"}]} {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 24 10:44:19.857778 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-ade08e2a-330f-46b2-9811-86dcec0930f3 req-27ae5b0e-faea-4234-9193-fae142fb011a service nova] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:19.858386 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-ade08e2a-330f-46b2-9811-86dcec0930f3 req-27ae5b0e-faea-4234-9193-fae142fb011a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:19.858724 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [req-ade08e2a-330f-46b2-9811-86dcec0930f3 req-27ae5b0e-faea-4234-9193-fae142fb011a service nova] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:19.871578 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.compute.server_external_events [req-ade08e2a-330f-46b2-9811-86dcec0930f3 req-27ae5b0e-faea-4234-9193-fae142fb011a service nova] Creating event network-vif-deleted:09fbe61e-3bab-42df-be0e-2c00c0b7934b for instance f51be366-af4a-492f-80b3-3c34b0f285ac on hv-cinder-85427 Aug 24 10:44:19.877228 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [req-ade08e2a-330f-46b2-9811-86dcec0930f3 req-27ae5b0e-faea-4234-9193-fae142fb011a service nova] 10.222.0.86 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036790 Aug 24 10:44:19.877938 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1551/3232] 10.222.0.86 () {64 vars in 1317 bytes} [Wed Aug 24 10:44:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 24 10:44:20.677214 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:20.677854 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:20.684653 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:20.685144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:20.685618 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:20.686313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:20.686769 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:20.687195 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:20.702621 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac HTTP/1.1 Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:44:20.704322 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:20.705597 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': 'f51be366-af4a-492f-80b3-3c34b0f285ac'} Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:44:20.706261 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:20.707564 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:20.708852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:44:20.708852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-a70682d3-465c-439d-aec3-13c34ed93f34 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 404 len: 111 microversion: 2.1 time: 0.032972 Aug 24 10:44:20.708852 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1682/3233] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:44:20 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:44:20.715508 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-d05217c0-d76b-47e9-932d-5c6979a00f74 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '.version_select at 0x7fa33db41f70>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:20.734017 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-d05217c0-d76b-47e9-932d-5c6979a00f74 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-164013944" status: 202 len: 0 microversion: 2.1 time: 0.022523 Aug 24 10:44:20.734512 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1552/3234] 10.222.0.86 () {58 vars in 1218 bytes} [Wed Aug 24 10:44:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-164013944 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 24 10:44:22.491019 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:22.491557 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:22.497907 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:22.498396 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:22.498859 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:22.499529 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:22.499988 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:22.500465 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:22.515543 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac HTTP/1.1 Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:44:22.517144 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.518484 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': 'f51be366-af4a-492f-80b3-3c34b0f285ac'} Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:44:22.519126 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.520389 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.521976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:44:22.521976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3513bf68-ab89-479d-9344-8db72c80a8ef tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 404 len: 111 microversion: 2.1 time: 0.031340 Aug 24 10:44:22.521976 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1683/3235] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:44:22 2022] DELETE /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:44:22.528331 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:22.529059 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: f51be366-af4a-492f-80b3-3c34b0f285ac] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:22.535645 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:22.536157 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:22.536631 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:22.537310 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:22.537760 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:22.538213 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:22.551964 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac HTTP/1.1 Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:44:22.553687 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.555107 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': 'f51be366-af4a-492f-80b3-3c34b0f285ac'} Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:44:22.555893 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. Aug 24 10:44:22.557099 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:22.558036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance f51be366-af4a-492f-80b3-3c34b0f285ac could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:44:22.558036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-88775af8-492f-4ba3-8d8d-cddbdc7ef2e6 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac" status: 404 len: 111 microversion: 2.1 time: 0.031755 Aug 24 10:44:22.558036 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1553/3236] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:44:22 2022] GET /compute/v2.1/servers/f51be366-af4a-492f-80b3-3c34b0f285ac => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:44:23.691855 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:23.692380 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:23.698665 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:23.699146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:23.699566 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:23.700272 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:23.700724 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:23.701146 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:23.714648 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c HTTP/1.1 Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:44:23.716204 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.717555 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '10a8d89c-8dad-488f-a9e3-4e8455592b4c'} Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:44:23.718441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.719800 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.721441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:44:23.721441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-f212ceb4-d7d5-4c4c-9005-3b8d09403155 tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 404 len: 111 microversion: 2.1 time: 0.029655 Aug 24 10:44:23.721441 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1684/3237] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:44:23 2022] DELETE /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:44:23.726932 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:23.727545 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 10a8d89c-8dad-488f-a9e3-4e8455592b4c] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:23.733954 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:23.734438 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:23.734874 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:23.735558 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:23.736010 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:23.736459 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:23.749522 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c HTTP/1.1 Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:44:23.751042 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.752453 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '10a8d89c-8dad-488f-a9e3-4e8455592b4c'} Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:44:23.753291 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. Aug 24 10:44:23.754632 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.756492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance 10a8d89c-8dad-488f-a9e3-4e8455592b4c could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:44:23.756492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-0b12d276-3859-4e6c-bfe5-66f9e859951a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c" status: 404 len: 111 microversion: 2.1 time: 0.030130 Aug 24 10:44:23.756492 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1554/3238] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:44:23 2022] GET /compute/v2.1/servers/10a8d89c-8dad-488f-a9e3-4e8455592b4c => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:44:23.761215 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:23.761716 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:23.767787 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:23.768426 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:23.768942 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:23.769767 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:23.770299 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:23.771473 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:23.785969 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 HTTP/1.1 Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:44:23.787561 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.789246 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '8af08746-c7d3-4ce2-8a80-6c322bd3a181'} Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:44:23.790193 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.791530 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.793234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:44:23.793234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-3dca429e-e8a2-418c-a618-843a159a2a3e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 404 len: 111 microversion: 2.1 time: 0.031951 Aug 24 10:44:23.793234 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1685/3239] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:44:23 2022] DELETE /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:44:23.798084 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:23.798718 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 8af08746-c7d3-4ce2-8a80-6c322bd3a181] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:23.805064 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:23.805559 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:23.806040 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:23.806732 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:23.807189 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:23.807614 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:23.821889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 HTTP/1.1 Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:44:23.823414 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.824961 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '8af08746-c7d3-4ce2-8a80-6c322bd3a181'} Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:44:23.825886 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. Aug 24 10:44:23.827271 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:23.828621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance 8af08746-c7d3-4ce2-8a80-6c322bd3a181 could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:44:23.828621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-b4cb0cc7-8df3-409c-953e-89dc7d733a0a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181" status: 404 len: 111 microversion: 2.1 time: 0.030627 Aug 24 10:44:23.828621 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1555/3240] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:44:23 2022] GET /compute/v2.1/servers/8af08746-c7d3-4ce2-8a80-6c322bd3a181 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 24 10:44:24.964559 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:24.965066 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.compute.api [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120653) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:24.971313 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:24.971728 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:24.972205 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:24.972887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:24.973335 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:24.973757 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG oslo_concurrency.lockutils [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120653) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:24.991596 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.wsgi [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: request: DELETE /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee HTTP/1.1 Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept: application/json Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Accept-Encoding: identity Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Connection: close Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Length: 0 Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Content-Type: application/json Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Host: 10.222.0.86 Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Openstack-System-Scope: None Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: User-Agent: python-urllib3/1.26.11 Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Id: None Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Domain-Name: None Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Identity-Status: Confirmed Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Is-Admin-Project: True Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Id: default Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Domain-Name: Default Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Role: reader,member Aug 24 10:44:24.993244 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Roles: reader,member Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Id: default Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Domain-Name: Default Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: method: > Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: exception: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:24.994848 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: action_args: {'id': '09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee'} Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: {{(pid=120653) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:44:24.995837 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:24.997279 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: ERROR nova.api.openstack.wsgi Aug 24 10:44:24.998839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: DEBUG nova.api.openstack.wsgi [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. {{(pid=120653) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:44:24.998839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO nova.api.openstack.requestlog [None req-8372692a-e4ad-4806-b69b-f2c30bb2088a tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "DELETE /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 404 len: 111 microversion: 2.1 time: 0.034450 Aug 24 10:44:24.998839 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: [pid: 120653|app: 0|req: 1686/3241] 10.222.0.86 () {58 vars in 1239 bytes} [Wed Aug 24 10:44:24 2022] DELETE /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:44:25.003221 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Calling method '>' {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 24 10:44:25.003839 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.compute.api [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] [instance: 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee] Fetching instance by UUID {{(pid=120654) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 24 10:44:25.010297 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:25.010770 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:25.011201 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:25.011881 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Acquiring lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 24 10:44:25.012398 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 24 10:44:25.012820 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG oslo_concurrency.lockutils [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Lock "62974e32-dd59-452e-afe9-bdb8e3ca6ae3" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120654) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 24 10:44:25.026435 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.wsgi [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] HTTP exception thrown: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Request method failure captured: Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: request: GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee HTTP/1.1 Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept: application/json Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Accept-Encoding: identity Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Connection: close Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Length: 0 Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Content-Type: application/json Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Host: 10.222.0.86 Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Openstack-System-Scope: None Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: User-Agent: python-urllib3/1.26.11 Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Auth-Token: gAAAAABjBf_3U3-wtHaQ5E_yOTRUb7yq_4ZgaEFxztnQi0H_3h0RPc7ByNZO1azpLQbHbrrbuEcGNY_iUqijgPSvURMA6Qq6_w2OW2Jfb2zwThe7kQEZDGCEBDc4fVR4SH_zMTqSx7pDZggF_IOkVKE7te30GGORtKg9ZAR-w88tLM4EPq3l814 Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Id: None Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Domain-Name: None Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Identity-Status: Confirmed Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Is-Admin-Project: True Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Id: default Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Domain-Name: Default Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Project-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Role: reader,member Aug 24 10:44:25.027979 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Roles: reader,member Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/image"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/compute/v2/9569b373f21841e8883b0b269073121a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.86:8080", "publicURL": "http://10.222.0.86:8080/v1/AUTH_9569b373f21841e8883b0b269073121a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86/volume/v3/9569b373f21841e8883b0b269073121a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.86:9696/networking"}]}] Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Id: 9569b373f21841e8883b0b269073121a Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-585830123 Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Id: default Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Domain-Name: Default Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Id: 311b0a8b22174a278793537963b5d142 Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: X-User-Name: tempest-AttachVolumeNegativeTest-585830123-project Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: method: > Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: exception: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:25.029499 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: action_args: {'id': '09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee'} Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: {{(pid=120654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 24 10:44:25.030343 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. Aug 24 10:44:25.031622 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: ERROR nova.api.openstack.wsgi Aug 24 10:44:25.032943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: DEBUG nova.api.openstack.wsgi [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] Returning 404 to user: Instance 09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee could not be found. {{(pid=120654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 24 10:44:25.032943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO nova.api.openstack.requestlog [None req-1588cffb-d693-47eb-8a5a-79b2768a798e tempest-AttachVolumeNegativeTest-585830123 tempest-AttachVolumeNegativeTest-585830123-project] 10.222.0.86 "GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee" status: 404 len: 111 microversion: 2.1 time: 0.030009 Aug 24 10:44:25.032943 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: [pid: 120654|app: 0|req: 1556/3242] 10.222.0.86 () {58 vars in 1236 bytes} [Wed Aug 24 10:44:24 2022] GET /compute/v2.1/servers/09dd0efc-7d2e-4f8d-87bd-06feff8ce3ee => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 24 10:44:50.700197 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:44:50.700875 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:44:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:46:15.288241 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:46:15.288887 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:46:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:46:27.010267 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:46:27.010836 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:46:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:46:32.137579 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:46:32.138086 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:46:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:46:44.288607 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:46:44.289334 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:46:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:48:10.708164 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:48:10.708763 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:48:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:49:35.296185 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:49:35.296802 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:49:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:49:47.018349 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:49:47.018947 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:49:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:49:52.145778 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:49:52.146310 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:49:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:50:04.296156 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:50:04.296796 dv-cinder-854270-2-iscsi devstack@n-api.service[120653]: Wed Aug 24 10:50:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 24 10:50:08.790889 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 24 10:50:08.791330 dv-cinder-854270-2-iscsi devstack@n-api.service[120654]: Wed Aug 24 10:50:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!